Medical support arm system and control device

ABSTRACT

Provision of a technology capable of realizing estimation of forces of various types of disturbance acting on a robot arm in a surgical environment is desirable. Provided is a medical support arm system including a joint state acquisition unit configured to acquire a state of a joint unit of an arm unit, and an external force estimation unit configured to estimate an external force due to predetermined disturbance on the basis of a condition that the external force due to the predetermined disturbance is limited to one predetermined direction or a plurality of predetermined directions, and a state of the joint unit.

TECHNICAL FIELD

The present disclosure relates to a medical support arm system and acontrol device.

BACKGROUND ART

Conventionally, for example, Patent Document 1 describes, in a medicalobservation device, a configuration including an imaging unit thatcaptures an image of an operation site, and a holding unit to which theimaging unit is connected and provided with rotation axes in an operablemanner with at least six degrees of freedom, in which at least two axes,of the rotation axes, are active axes controlled to be driven on thebasis of states of the rotation axes, and at least one axis, of therotation axes, is a passive axis rotated according to a direct operationwith contact from an outside.

CITATION LIST Patent Document

-   Patent Document 1: International Publication No. 2016/017532

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

By the way, an arm intended for surgery support is used in anenvironment where various types of disturbance act. However, it isgenerally difficult to estimate the force acting from the disturbanceregardless of conditions such as an environment and a scene.

Therefore, provision of a technology capable of realizing estimation offorces of various types of disturbance acting on an arm in a surgicalenvironment is desirable.

Solutions to Problems

According to the present disclosure, provided is a medical support armsystem including a joint state acquisition unit configured to acquire astate of a joint unit of an arm unit, and an external force estimationunit configured to estimate an external force due to predetermineddisturbance on the basis of a condition that the external force due tothe predetermined disturbance is limited to one predetermined directionor a plurality of predetermined directions, and a state of the jointunit.

Effects of the Invention

According to the above-described present disclosure, estimation offorces of various types of disturbance acting on an arm in a surgicalenvironment can be realized.

Note that the above-described effect is not necessarily limited, and anyof effects described in the present specification or other effects thatcan be grasped from the present specification may be exerted in additionto or in place of the above-described effect.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an example of a schematic configurationof an endoscopic surgical system to which the technology according tothe present disclosure is applicable.

FIG. 2 is a block diagram illustrating an example of functionalconfigurations of a camera head and a CCU illustrated in FIG. 1.

FIG. 3 is a perspective view illustrating a configuration example of amedical support arm device according to an embodiment of the presentdisclosure.

FIG. 4 is an explanatory diagram for describing ideal joint controlaccording to an embodiment of the present disclosure.

FIG. 5 is a functional block diagram illustrating a configurationexample of an arm control system according to an embodiment of thepresent disclosure.

FIG. 6 is a view illustrating an example of a schematic configuration ofa microsurgical system to which the technology according to the presentdisclosure is applicable.

FIG. 7 is a view illustrating an appearance of a hard endoscope unit.

FIG. 8 is an enlarged view of a connection portion.

FIG. 9 is a view for describing an example of a force acting from atrocar point.

FIG. 10 is a view for describing an example of joint control in a casewhere an observation point is placed at a distal end of the hardendoscope.

FIG. 11 is a view for describing an example of joint control in a casewhere an observation point is placed at a distal end of the hardendoscope.

FIG. 12 is a diagram illustrating a specific configuration example ofthe arm control system.

MODE FOR CARRYING OUT THE INVENTION

Favorable embodiments of the present disclosure will be described indetail with reference to the appended drawings. Note that, in thepresent specification and drawings, redundant description ofconfiguration elements having substantially the same functionalconfiguration is omitted by providing the same sign.

Note that the description will be given in the following order.

1. Configuration Example of Endoscopic System

2. Specific Configuration Example of Support Arm Device

3. Example of configuration of Microsurgical System

4. Estimation of Force Acting from Disturbance According to PresentEmbodiment

5. Joint Control According to External Force According to PresentEmbodiment

6. Specific Configuration Example of Arm Control System

7. Conclusion

1. Configuration Example of Endoscopic System

FIG. 1 is a diagram illustrating an example of a schematic configurationof an endoscopic surgical system 5000 to which the technology accordingto the present disclosure is applicable. FIG. 1 illustrates a state inwhich an operator (surgeon) 5067 is performing an operation on a patient5071 on a patient bed 5069, using the endoscopic surgical system 5000.As illustrated, the endoscopic surgical system 5000 includes anendoscope 5001, other surgical tools 5017, a support arm device 5027that supports the endoscope 5001, and a cart 5037 in which variousdevices for endoscopic surgery are mounted.

In an endoscopic surgery, a plurality of cylindrical punctureinstruments called trocars 5025 a to 5025 d is punctured into anabdominal wall instead of cutting the abdominal wall and opening theabdomen. Then, a lens barrel 5003 of the endoscope 5001 and othersurgical tools 5017 are inserted into a body cavity of the patient 5071through the trocars 5025 a to 5025 d. In the illustrated example, as theother surgical tools 5017, a pneumoperitoneum tube 5019, an energytreatment tool 5021, and a forceps 5023 are inserted into the bodycavity of the patient 5071. Furthermore, the energy treatment tool 5021is a treatment tool for performing incision and detachment of tissue,sealing of a blood vessel, and the like with a high-frequency current oran ultrasonic vibration. Note that the illustrated surgical tools 5017are mere examples, and various kinds of surgical tools typically used inendoscopic surgery such as tweezers and a retractor may be used as thesurgical tool 5017.

An image of an operation site in the body cavity of the patient 5071captured by the endoscope 5001 is displayed on a display device 5041.The operator 5067 performs treatment such as removal of an affectedpart, using the energy treatment tool 5021 and the forceps 5023 whileviewing the image of the operation site displayed on the display device5041 in real time. Note that the pneumoperitoneum tube 5019, the energytreatment tool 5021, and the forceps 5023 are supported by the operator5067, an assistant, or the like during surgery, although illustration isomitted.

(Support Arm Device)

The support arm device 5027 includes an arm unit 5031 extending from abase unit 5029. In the illustrated example, the arm unit 5031 includesjoint units 5033 a, 5033 b, and 5033 c, and links 5035 a and 5035 b, andis driven under the control of an arm control device 5045. The endoscope5001 is supported by the arm unit 5031, and the position and posture ofthe endoscope 5001 are controlled. With the control, stable fixation ofthe position of the endoscope 5001 can be realized.

(Endoscope)

The endoscope 5001 includes the lens barrel 5003 and a camera head 5005.A region having a predetermined length from a distal end of the lensbarrel 5003 is inserted into the body cavity of the patient 5071. Thecamera head 5005 is connected to a proximal end of the lens barrel 5003.In the illustrated example, the endoscope 5001 configured as a so-calledhard endoscope including the hard lens barrel 5003 is illustrated.

However, the endoscope 5001 may be configured as a so-called softendoscope including the soft lens barrel 5003.

An opening portion in which an object lens is fit is provided in thedistal end of the lens barrel 5003. A light source device 5043 isconnected to the endoscope 5001, and light generated by the light sourcedevice 5043 is guided to the distal end of the lens barrel 5003 by alight guide extending inside the lens barrel 5003 and an observationtarget in the body cavity of the patient 5071 is irradiated with thelight through the object lens. Note that the endoscope 5001 may be adirect-viewing endoscope, an oblique-viewing endoscope, or aside-viewing endoscope.

An optical system and an imaging element are provided inside the camerahead 5005, and reflected light (observation light) from the observationtarget is condensed to the imaging element by the optical system. Theobservation light is photoelectrically converted by the imaging element,and an electrical signal corresponding to the observation light, thatis, an image signal corresponding to an observed image is generated. Theimage signal is transmitted to a camera control unit (CCU) 5039 as rawdata. Note that the camera head 5005 has a function to adjustmagnification and a focal length by appropriately driving the opticalsystem.

Note that a plurality of the imaging elements may be provided in thecamera head 5005 to support three-dimensional (3D) display, and thelike, for example. In this case, a plurality of relay optical systems isprovided inside the lens barrel 5003 to guide the observation light toeach of the plurality of imaging elements.

(Various Devices Mounted in Cart)

The CCU 5039 includes a central processing unit (CPU), a graphicsprocessing unit (GPU), and the like, and centrally controls theoperation of the endoscope 5001 and the display device 5041.Specifically, the CCU 5039 receives the image signal from the camerahead 5005, and applies various types of image processing for displayingan image based on the image signal, such as developing processing(demosaicing processing), for example, to the image signal. The CCU 5039provides the image signal to which the image processing has been appliedto the display device 5041. Furthermore, the CCU 5039 transmits acontrol signal to the camera head 5005 to control its driving. Thecontrol signal may include information regarding imaging conditions suchas the magnification and focal length.

The display device 5041 displays an image based on the image signal towhich the image processing has been applied by the CCU 5039, under thecontrol of the CCU 5039. In a case where the endoscope 5001 supportshigh-resolution capturing such as 4K (horizontal pixel number3840×vertical pixel number 2160) or 8K (horizontal pixel number7680×vertical pixel number 4320), and/or in a case where the endoscope5001 supports 3D display, for example, the display device 5041, whichcan perform high-resolution display and/or 3D display, can be usedcorresponding to each case. In a case where the endoscope 5001 supportsthe high-resolution capturing such as 4K or 8K, a greater sense ofimmersion can be obtained by use of the display device 5041 with thesize of 55 inches or more. Furthermore, a plurality of display devices5041 having different resolutions and sizes may be provided depending onthe application.

The light source device 5043 includes a light source such as a lightemitting diode (LED) for example, and supplies irradiation light to theendoscope 5001 in capturing an operation portion.

The arm control device 5045 includes a processor such as a CPU, and isoperated according to a predetermined program, thereby to controldriving of the arm unit 5031 of the support arm device 5027 according toa predetermined control method.

An input device 5047 is an input interface for the endoscopic surgicalsystem 5000. The user can input various types of information andinstructions to the endoscopic surgical system 5000 through the inputdevice 5047. For example, the user inputs various types of informationregarding surgery, such as patient's physical information andinformation of an operative procedure of the surgery, through the inputdevice 5047. Furthermore, for example, the user inputs an instruction todrive the arm unit 5031, an instruction to change the imaging conditions(such as the type of the irradiation light, the magnification, and thefocal length) of the endoscope 5001, an instruction to drive the energytreatment tool 5021, or the like through the input device 5047.

The type of the input device 5047 is not limited, and the input device5047 may be one of various known input devices. For example, a mouse, akeyboard, a touch panel, a switch, a foot switch 5057, and/or a levercan be applied to the input device 5047. In a case where a touch panelis used as the input device 5047, the touch panel may be provided on adisplay surface of the display device 5041.

Alternatively, the input device 5047 is a device worn by the user, suchas a glass-type wearable device or a head mounted display (HMD), forexample, and various inputs are performed according to a gesture or aline of sight of the user detected by the device. Furthermore, the inputdevice 5047 includes a camera capable of detecting a movement of theuser, and various inputs are performed according to a gesture or a lineof sight of the user detected from a video captured by the camera.Moreover, the input device 5047 includes a microphone capable ofcollecting a voice of the user, and various inputs are performed by anaudio through the microphone. In this way, the input device 5047 isconfigured to be able to input various types of information in anon-contact manner, whereby the user (for example, the operator 5067) inparticular belonging to a clean area can operate a device belonging to afilthy area in a non-contact manner. Furthermore, since the user canoperate the device without releasing his/her hand from the possessedsurgical tool, the user's convenience is improved.

A treatment tool control device 5049 controls driving of the energytreatment tool 5021 for cauterization and incision of tissue, sealing ofa blood vessel, and the like. A pneumoperitoneum device 5051 sends a gasinto the body cavity of the patient 5071 through the pneumoperitoneumtube 5019 to expand the body cavity for the purpose of securing a fieldof view by the endoscope 5001 and a work space for the operator. Arecorder 5053 is a device that can record various types of informationregarding the surgery. A printer 5055 is a device that can print thevarious types of information regarding the surgery in various formatssuch as a text, an image, or a graph.

Hereinafter, a particularly characteristic configuration in theendoscopic surgical system 5000 will be further described in detail.

(Support Arm Device)

The support arm device 5027 includes the base unit 5029 as a base andthe arm unit 5031 extending from the base unit 5029. In the illustratedexample, the arm unit 5031 includes the plurality of joint units 5033 a,5033 b, and 5033 c and the plurality of links 5035 a and 5035 bconnected by the joint unit 5033 b, but FIG. 1 illustrates theconfiguration of the arm unit 5031 in a simplified manner forsimplification. In reality, the shapes, the number, and the arrangementof the joint units 5033 a to 5033 c and the links 5035 a and 5035 b, thedirections of rotation axes of the joint units 5033 a to 5033 c, and thelike can be appropriately set so that the arm unit 5031 has a desireddegree of freedom. For example, the arm unit 5031 can be favorablyconfigured to have six degrees of freedom or more. With theconfiguration, the endoscope 5001 can be freely moved within a movablerange of the arm unit 5031. Therefore, the lens barrel 5003 of theendoscope 5001 can be inserted from a desired direction into the bodycavity of the patient 5071.

Actuators are provided in the joint units 5033 a to 5033 c, and thejoint units 5033 a to 5033 c are configured to be rotatable around apredetermined rotation axis by driving of the actuators. The driving ofthe actuators is controlled by the arm control device 5045, wherebyrotation angles of the joint units 5033 a to 5033 c are controlled anddriving of the arm unit 5031 is controlled. With the control, control ofthe position and posture of the endoscope 5001 can be realized. At thistime, the arm control device 5045 can control the driving of the armunit 5031 by various known control methods such as force control orposition control.

For example, the driving of the arm unit 5031 may be appropriatelycontrolled by the arm control device 5045 according to an operationinput, and the position and posture of the endoscope 5001 may becontrolled, by an appropriate operation input by the operator 5067 viathe input device 5047 (including the foot switch 5057). With thecontrol, the endoscope 5001 at the distal end of the arm unit 5031 canbe moved from an arbitrary position to an arbitrary position, and thencan be fixedly supported at the position after the movement. Note thatthe arm unit 5031 may be operated by a so-called master-slave system. Inthis case, the arm unit 5031 can be remotely operated by the user viathe input device 5047 installed at a place distant from an operatingroom.

Furthermore, in a case where the force control is applied, the armcontrol device 5045 may perform so-called power assist control in whichthe arm control device 5045 receives an external force from the user anddrives the actuators of the joint units 5033 a to 5033 c so that the armunit 5031 is smoothly moved according to the external force. With thecontrol, the user can move the arm unit 5031 with a relatively lightforce when moving the arm unit 5031 while being in direct contact withthe arm unit 5031. Accordingly, the user can more intuitively move theendoscope 5001 with a simpler operation, and the user's convenience canbe improved.

Here, in endoscopic surgery, the endoscope 5001 has been generallysupported by a surgeon called scopist. In contrast, by use of thesupport arm device 5027, the position of the endoscope 5001 can bereliably fixed without manual operation, and thus an image of theoperation site can be stably obtained and the surgery can be smoothlyperformed.

Note that the arm control device 5045 is not necessarily provided in thecart 5037. Furthermore, the arm control device 5045 is not necessarilyone device. For example, the arm control device 5045 may be provided ineach of the joint units 5033 a to 5033 c of the arm unit 5031 of thesupport arm device 5027, and the drive control of the arm unit 5031 maybe realized by mutual cooperation of the plurality of arm controldevices 5045.

(Light Source Device)

The light source device 5043 supplies irradiation light, which is usedin capturing an operation site, to the endoscope 5001. The light sourcedevice 5043 includes, for example, an LED, a laser light source, or awhite light source configured by a combination thereof. In a case wherethe white light source is configured by a combination of RGB laser lightsources, output intensity and output timing of the respective colors(wavelengths) can be controlled with high accuracy. Therefore, whitebalance of a captured image can be adjusted in the light source device5043. Further, in this case, the observation target is irradiated withthe laser light from each of the RGB laser light sources in a timedivision manner, and the driving of the imaging element of the camerahead 5005 is controlled in synchronization with the irradiation timing,so that images respectively corresponding to RGB can be captured in atime division manner. According to the method, a color image can beobtained without providing a color filter to the imaging element.

Furthermore, driving of the light source device 5043 may be controlledto change intensity of light to be output every predetermined time. Thedriving of the imaging element of the camera head 5005 is controlled insynchronization with change timing of the intensity of light, and imagesare acquired in a time division manner and are synthesized, whereby ahigh-dynamic range image without clipped blacks and flared highlightscan be generated.

Further, the light source device 5043 may be configured to be able tosupply light in a predetermined wavelength band corresponding to speciallight observation. In the special light observation, for example,so-called narrow band imaging is performed by radiating light in anarrower band than the irradiation light (in other words, white light)at the time of normal observation, using wavelength dependence ofabsorption of light in a body tissue, to capture a predetermined tissuesuch as a blood vessel in a mucosal surface layer at high contrast.Alternatively, in the special light observation, fluorescenceobservation to obtain an image by fluorescence generated by radiation ofexciting light may be performed. In the fluorescence observation,irradiating the body tissue with exciting light to observe fluorescencefrom the body tissue (self-fluorescence observation), injecting areagent such as indocyanine green (ICG) into the body tissue andirradiating the body tissue with exciting light corresponding to afluorescence wavelength of the reagent to obtain a fluorescence image,or the like can be performed. The light source device 5043 can beconfigured to be able to supply narrow-band light and/or exciting lightcorresponding to such special light observation.

(Camera Head and CCU)

Functions of the camera head 5005 and the CCU 5039 of the endoscope 5001will be described in more detail with reference to FIG. 2. FIG. 2 is ablock diagram illustrating an example of functional configurations ofthe camera head 5005 and the CCU 5039 illustrated in FIG. 1.

Referring to FIG. 2, the camera head 5005 includes a lens unit 5007, animaging unit 5009, a drive unit 5011, a communication unit 5013, and acamera head control unit 5015 as its functions. Furthermore, the CCU5039 includes a communication unit 5059, an image processing unit 5061,and a control unit 5063 as its functions. The camera head 5005 and theCCU 5039 are communicatively connected with each other by a transmissioncable 5065.

First, a functional configuration of the camera head 5005 will bedescribed. The lens unit 5007 is an optical system provided in aconnection portion between the lens unit 5007 and the lens barrel 5003.Observation light taken through the distal end of the lens barrel 5003is guided to the camera head 5005 and enters the lens unit 5007. Thelens unit 5007 is configured by a combination of a plurality of lensesincluding a zoom lens and a focus lens. Optical characteristics of thelens unit 5007 are adjusted to condense the observation light on a lightreceiving surface of an imaging element of the imaging unit 5009.Furthermore, the zoom lens and the focus lens are configured to havetheir positions on the optical axis movable for adjustment of themagnification and focal point of the captured image.

The imaging unit 5009 includes an imaging element, and is disposed at arear stage of the lens unit 5007. The observation light having passedthrough the lens unit 5007 is focused on the light receiving surface ofthe imaging element, and an image signal corresponding to the observedimage is generated by photoelectric conversion. The image signalgenerated by the imaging unit 5009 is provided to the communication unit5013.

As the imaging element constituting the imaging unit 5009, for example,a complementary metal oxide semiconductor (CMOS)-type image sensorhaving Bayer arrangement and capable of color capturing is used. Notethat, as the imaging element, for example, an imaging element that cancapture a high-resolution image of 4K or more may be used. By obtainmentof the image of the operation site with high resolution, the operator5067 can grasp the state of the operation site in more detail and canmore smoothly advance the surgery.

Furthermore, the imaging element constituting the imaging unit 5009includes a pair of imaging elements for respectively obtaining imagesignals for right eye and for left eye corresponding to 3D display. Withthe 3D display, the operator 5067 can more accurately grasp the depth ofbiological tissue in the operation site. Note that, in a case where theimaging unit 5009 is configured as a multi-plate imaging unit, aplurality of systems of the lens units 5007 is provided corresponding tothe imaging elements.

Furthermore, the imaging unit 5009 may not be necessarily provided inthe camera head 5005. For example, the imaging unit 5009 may be providedimmediately after the object lens inside the lens barrel 5003.

The drive unit 5011 includes an actuator, and moves the zoom lens andthe focus lens of the lens unit 5007 by a predetermined distance alongan optical axis by the control of the camera head control unit 5015.With the movement, the magnification and focal point of the capturedimage by the imaging unit 5009 can be appropriately adjusted.

The communication unit 5013 includes a communication device fortransmitting or receiving various types of information to or from theCCU 5039. The communication unit 5013 transmits the image signalobtained from the imaging unit 5009 to the CCU 5039 through thetransmission cable 5065 as raw data. At this time, to display thecaptured image of the operation site with low latency, the image signalis favorably transmitted by optical communication. This is because, insurgery, the operator 5067 performs surgery while observing the state ofthe affected part with the captured image, and thus display of a movingimage of the operation site in as real time as possible is demanded formore safe and reliable surgery. In the case of the opticalcommunication, a photoelectric conversion module that converts anelectrical signal into an optical signal is provided in thecommunication unit 5013. The image signal is converted into the opticalsignal by the photoelectric conversion module, and is then transmittedto the CCU 5039 via the transmission cable 5065.

Furthermore, the communication unit 5013 receives a control signal forcontrolling driving of the camera head 5005 from the CCU 5039. Thecontrol signal includes information regarding the imaging conditionssuch as information for specifying a frame rate of the captured image,information for specifying an exposure value at the time of imaging,and/or information for specifying the magnification and the focal pointof the captured image, for example. The communication unit 5013 providesthe received control signal to the camera head control unit 5015. Notethat the control signal from that CCU 5039 may also be transmitted bythe optical communication. In this case, the communication unit 5013 isprovided with a photoelectric conversion module that converts an opticalsignal into an electrical signal, and the control signal is convertedinto an electrical signal by the photoelectric conversion module and isthen provided to the camera head control unit 5015.

Note that the imaging conditions such as the frame rate, exposure value,magnification, and focal point are automatically set by the control unit5063 of the CCU 5039 on the basis of the acquired image signal. That is,a so-called auto exposure (AE) function, an auto focus (AF) function,and an auto white balance (AWB) function are incorporated in theendoscope 5001.

The camera head control unit 5015 controls the driving of the camerahead 5005 on the basis of the control signal received from the CCU 5039through the communication unit 5013. For example, the camera headcontrol unit 5015 controls driving of the imaging element of the imagingunit 5009 on the basis of the information for specifying the frame rateof the captured image and/or the information for specifying exposure atthe time of imaging. Furthermore, for example, the camera head controlunit 5015 appropriately moves the zoom lens and the focus lens of thelens unit 5007 via the drive unit 5011 on the basis of the informationfor specifying the magnification and focal point of the captured image.The camera head control unit 5015 may further have a function to storeinformation for identifying the lens barrel 5003 and the camera head5005.

Note that the configuration of the lens unit 5007, the imaging unit5009, and the like is arranged in a hermetically sealed structure havinghigh airtightness and waterproofness, whereby the camera head 5005 canhave resistance to autoclave sterilization processing.

Next, a functional configuration of the CCU 5039 will be described. Thecommunication unit 5059 includes a communication device for transmittingor receiving various types of information to or from the camera head5005. The communication unit 5059 receives the image signal transmittedfrom the camera head 5005 through the transmission cable 5065. At thistime, as described above, the image signal can be favorably transmittedby the optical communication. In this case, the communication unit 5059is provided with a photoelectric conversion module that converts anoptical signal into an electrical signal, corresponding to the opticalcommunication. The communication unit 5059 provides the image signalconverted into the electrical signal to the image processing unit 5061.

Furthermore, the communication unit 5059 transmits a control signal forcontrolling driving of the camera head 5005 to the camera head 5005. Thecontrol signal may also be transmitted by the optical communication.

The image processing unit 5061 applies various types of image processingto the image signal as raw data transmitted from the camera head 5005.The image processing include various types of known signal processingsuch as development processing, high image quality processing (such asband enhancement processing, super resolution processing, noisereduction (NR) processing, and/or camera shake correction processing),and/or enlargement processing (electronic zoom processing), for example.Furthermore, the image processing unit 5061 performs wave detectionprocessing for image signals for performing AE, AF, and AWB.

The image processing unit 5061 is configured by a processor such as aCPU or a GPU, and the processor is operated according to a predeterminedprogram, whereby the above-described image processing and wave detectionprocessing can be performed. Note that in a case where the imageprocessing unit 5061 includes a plurality of GPUs, the image processingunit 5061 appropriately divides the information regarding the imagesignal and performs the image processing in parallel by the plurality ofGPUs.

The control unit 5063 performs various types of control related toimaging of the operation site by the endoscope 5001 and display of thecaptured image. For example, the control unit 5063 generates a controlsignal for controlling driving of the camera head 5005. At this time, ina case where the imaging conditions are input by the user, the controlunit 5063 generates the control signal on the basis of the input by theuser. Alternatively, in a case where the AE function, the AF function,and the AWB function are incorporated in the endoscope 5001, the controlunit 5063 appropriately calculates optimum exposure value, focal length,and white balance according to a result of the wave detection processingby the image processing unit 5061, and generates the control signal.

Furthermore, the control unit 5063 displays the image of the operationsite on the display device 5041 on the basis of the image signal towhich the image processing has been applied by the image processing unit5061. At this time, the control unit 5063 recognizes various objects inthe image of the operation site, using various image recognitiontechnologies. For example, the control unit 5063 can recognize asurgical instrument such as forceps, a specific living body portion,blood, mist at the time of use of the energy treatment tool 5021, or thelike, by detecting a shape of an edge, a color or the like of an objectincluded in the operation site image. The control unit 5063 superimposesand displays various types of surgery support information on the imageof the operation site, in displaying the image of the operation site onthe display device 5041 using the result of recognition. The surgerysupport information is superimposed, displayed, and presented to theoperator 5067, so that the surgery can be more safely and reliablyadvanced.

The transmission cable 5065 that connects the camera head 5005 and theCCU 5039 is an electrical signal cable supporting communication ofelectrical signals, an optical fiber supporting optical communication,or a composite cable thereof.

Here, in the illustrated example, the communication has been performedin a wired manner using the transmission cable 5065. However, thecommunication between the camera head 5005 and the CCU 5039 may bewirelessly performed. In a case where the communication between thecamera head 5005 and the CCU 5039 is wirelessly performed, it isunnecessary to lay the transmission cable 5065 in the operating room.Therefore, the situation in which movement of medical staffs in theoperating room is hindered by the transmission cable 5065 can beeliminated.

An example of an endoscopic surgical system 5000 to which the technologyaccording to the present disclosure is applicable has been described.Note that, here, the endoscopic surgical system 5000 has been describedas an example. However, a system to which the technology according tothe present disclosure is applicable is not limited to this example. Forexample, the technique according to the present disclosure may beapplied to a flexible endoscopic system for examination or amicrosurgical system.

2. Specific Configuration Example of Support Arm Device

Next, a specific configuration example of a support arm device accordingto the embodiment of the present disclosure will be described in detail.The support arm device described below is an example configured as asupport arm device that supports an endoscope at a distal end of an armunit. However, the present embodiment is not limited to the example.

Furthermore, in a case where the support arm device according to theembodiment of the present disclosure is applied to the medical field,the support arm device according to the embodiment of the presentdisclosure can function as a medical support arm device.

<2-1. Appearance of Support Arm Device>

First, a schematic configuration of a support arm device 400 accordingto the present embodiment will be described with reference to FIG. 3.FIG. 3 is a schematic view illustrating an appearance of the support armdevice 400 according to the present embodiment.

The support arm device 400 according to the present embodiment includesa base unit 410 and an arm unit 420. The base unit 410 is a base of thesupport arm device 400, and the arm unit 420 is extended from the baseunit 410. Furthermore, although not illustrated in FIG. 3, a controlunit that integrally controls the support arm device 400 may be providedin the base unit 410, and driving of the arm unit 420 may be controlledby the control unit. The control unit includes various signal processingcircuits, such as a CPU and a DSP, for example.

The arm unit 420 includes a plurality of active joint units 421 a to 421f, a plurality of links 422 a to 422 f, and an endoscope device 423 as adistal end unit provided at a distal end of the arm unit 420.

The links 422 a to 422 f are substantially rod-like members. One end ofthe link 422 a is connected to the base unit 410 via the active jointunit 421 a, the other end of the link 422 a is connected to one end ofthe link 422 b via the active joint unit 421 b, and the other end of thelink 422 b is connected to one end of the link 422 c via the activejoint unit 421 c. The other end of the link 422 c is connected to thelink 422 d via a passive slide mechanism 100, and the other end of thelink 422 d is connected to one end of the link 422 e via a passive jointunit 200. The other end of the link 422 e is connected to one end of thelink 422 f via the active joint units 421 d and 421 e. The endoscopedevice 423 is connected to the distal end of the arm unit 420, in otherwords, the other end of the link 422 f, via the active joint unit 421 f.The respective ends of the plurality of links 422 a to 422 f areconnected one another by the active joint units 421 a to 421 f, thepassive slide mechanism 100, and the passive joint unit 200 with thebase unit 410 as a fulcrum, as described above, so that an arm shapeextended from the base unit 410 is configured.

Actuators provided in the respective active joint units 421 a to 421 fof the arm unit 420 are driven and controlled, so that the position andposture of the endoscope device 423 are controlled. In the presentembodiment, the endoscope device 423 has a distal end enter a bodycavity of a patient, which is an operation site, and captures a partialregion of the operation site. Note that the distal end unit provided atthe distal end of the arm unit 420 is not limited to the endoscopedevice 423, and various medical instruments may be connected to thedistal end of the arm unit 420 as the distal end units. Thus, thesupport arm device 400 according to the present embodiment is configuredas a medical support arm device provided with a medical instrument.

Here, hereinafter, the support arm device 400 will be described bydefining coordinate axes as illustrated in FIG. 3. Furthermore, anup-down direction, a front-back direction, and a right-left directionwill be defined in accordance with the coordinate axes. In other words,the up-down direction with respect to the base unit 410 installed on afloor is defined as a z-axis direction and the up-down directionFurthermore, a direction orthogonal to the z axis and in which the armunit 420 is extended from the base unit 410 (in other words, a directionin which the endoscope device 423 is located with respect to the baseunit 410) is defined as a y-axis direction and the front-back direction.Moreover, a direction orthogonal to the y axis and the z axis is definedas an x-axis direction and the right-left direction.

The active joint units 421 a to 421 f rotatably connect the links to oneanother. The active joint units 421 a to 421 f include actuators, andhave a rotation mechanism that is rotationally driven about apredetermined rotation axis by driving of the actuators. By controllingrotational driving of each of the active joint units 421 a to 421 f,driving of the arm unit 420 such as extending or contracting (folding)of the arm unit 420 can be controlled. Here, the driving of the activejoint units 421 a to 421 f can be controlled by, for example, knownwhole body coordination control and ideal joint control. As describedabove, since the active joint units 421 a to 421 f have the rotationmechanism, in the following description, the drive control of the activejoint units 421 a to 421 f specifically means control of rotation anglesand/or generated torque (torque generated by the active joint units 421a to 421 f) of the active joint units 421 a to 421 f.

The passive slide mechanism 100 is an aspect of a passive form changemechanism, and connects the link 422 c and the link 422 d to be able tomove forward and backward along a predetermined direction. For example,the passive slide mechanism 100 may connect the link 422 c and the link422 d in a linearly movable manner. However, the forward/backward motionof the link 422 c and the link 422 d is not limited to the linearmotion, and may be forward/backward motion in a direction of forming anarc. The passive slide mechanism 100 is operated in the forward/backwardmotion by a user, for example, and makes a distance between the activejoint unit 421 c on the one end side of the link 422 c and the passivejoint unit 200 variable. Thereby, the entire form of the arm unit 420can change.

The passive joint unit 200 is one aspect of the passive form changemechanism, and rotatably connects the link 422 d and the link 422 e toeach other. The passive joint unit 200 is rotatably operated by theuser, for example, and makes an angle made by the link 422 d and thelink 422 e variable. Thereby, the entire form of the arm unit 420 canchange.

Note that, in the present specification, a “posture of the arm unit”refers to a state of the arm unit changeable by the drive control of theactuators provided in the active joint units 421 a to 421 f by thecontrol unit in a state where the distance between active joint unitsadjacent across one or a plurality of links is constant. Furthermore, a“form of the arm unit” refers to a state of the arm unit changeable asthe distance between active joint units adjacent across a link or anangle between links connecting adjacent active joint units changes withthe operation of the passive form change mechanism.

The support arm device 400 according to the present embodiment includesthe six active joint units 421 a to 421 f and realizes six degrees offreedom with respect to the driving of the arm unit 420. That is, whilethe drive control of the support arm device 400 is realized by the drivecontrol of the six active joint units 421 a to 421 f by the controlunit, the passive slide mechanism 100 and the passive joint unit 200 arenot the targets of the drive control by the control unit.

Specifically, as illustrated in FIG. 3, the active joint units 421 a,421 d, and 421 f are provided to have long axis directions of theconnected links 422 a and 422 e and a capture direction of the connectedendoscope device 423 as rotation axis directions. The active joint units421 b, 421 c, and 421 e are provided to have the x-axis direction thatis a direction in which connection angles of the connected links 422 ato 422 c, 422 e, and 422 f and the connected endoscope device 423 arechanged in a y-z plane (a plane defined by the y axis and the z axis) asrotation axis directions. As described above, in the present embodiment,the active joint units 421 a, 421 d, and 421 f have a function toperform so-called yawing, and the active joint units 421 b, 421 c, and421 e have a function to perform so-called pitching.

With the above configuration of the arm unit 420, the support arm device400 according to the present embodiment realizes the six degrees offreedom with respect to the driving of the arm unit 420, whereby freelymoving the endoscope device 423 within the movable range of the arm unit420. FIG. 3 illustrates a hemisphere as an example of a movable range ofthe endoscope device 423. In a case where a central point RCM (remotemotion center) of the hemisphere is a capture center of the operationsite captured by the endoscope device 423, the operation site can becaptured from various angles by moving the endoscope device 423 on aspherical surface of the hemisphere in a state where the capture centerof the endoscope device 423 is fixed to the central point of thehemisphere.

The schematic configuration of the support arm device 400 according tothe present embodiment has been described above. Next, the whole bodycoordination control and the ideal joint control for controlling thedriving of the arm unit 420, in other words, the driving of the jointunits 421 a to 421 f in the support arm device 400 according to thepresent embodiment will be described.

<2-2. Generalized Inverse Dynamics>

Next, an overview of generalized inverse dynamics used for the wholebody coordination control of the support arm device 400 in the presentembodiment will be described.

The generalized inverse dynamics is basic operation in the whole bodycoordination control of a multilink structure configured by connecting aplurality of links by a plurality of joint units (for example, the armunit 420 illustrated in FIG. 2 in the present embodiment), forconverting motion purposes regarding various dimensions in variousoperation spaces into torque to be caused in the plurality of jointunits in consideration of various constraint conditions.

The operation space is an important concept in force control of a robotdevice. The operation space is a space for describing a relationshipbetween force acting on the multilink structure and acceleration of themultilink structure. When the drive control of the multilink structureis performed not by position control but by force control, the conceptof the operation space is required in a case of using a contact betweenthe multilink structure and an environment as a constraint condition.The operation space is, for example, a joint space, a Cartesian space, amomentum space, or the like, which is a space to which the multilinkstructure belongs.

The motion purpose represents a target value in the drive control of themultilink structure, and is, for example, a target value of a position,a speed, an acceleration, a force, an impedance, or the like of themultilink structure to be achieved by the drive control.

The constraint condition is a constraint condition regarding theposition, speed, acceleration, force, or the like of the multilinkstructure, which is determined according to a shape or a structure ofthe multilink structure, an environment around the multilink structure,settings by the user, and the like. For example, the constraintcondition includes information regarding a generated force, a priority,presence/absence of a non-drive joint, a vertical reaction force, afriction weight, a support polygon, and the like.

In the generalized dynamics, to establish both stability of numericalcalculation and real time processing efficiency, an arithmetic algorithmincludes a virtual force determination process (virtual forcecalculation processing) as a first stage and a real force conversionprocess (real force calculation processing) as a second stage. In thevirtual force calculation processing as the first stage, a virtual forcethat is a virtual force required for achievement of each motion purposeand acting on the operation space is determined while considering thepriority of the motion purpose and a maximum value of the virtual force.In the real force calculation processing as the second stage, theabove-obtained virtual force is converted into a real force realizablein the actual configuration of the multilink structure, such as a jointforce or an external force, while considering the constraints regardingthe non-drive joint, the vertical reaction force, the friction weight,the support polygon, and the like. Hereinafter, the virtual forcecalculation processing and the real force calculation processing will bedescribed in detail. Note that, in the description of the virtual forcecalculation processing and the real force calculation processing belowand the real force calculation processing to be described below,description may be performed using the configuration of the arm unit 420of the support arm device 400 according to the present embodimentillustrated in FIG. 3 as a specific example, in order to facilitateunderstanding.

(2-2-1. Virtual Force Calculation Processing)

A vector configured by a certain physical quantity at each joint unit ofthe multilink structure is called generalized variable q (also referredto as a joint value q or a joint space q). An operation space x isdefined by the following expression (1) using a time derivative value ofthe generalized variable q and the Jacobian J.

[Math. 1]

{dot over (x)}=J{dot over (q)}  (1)

In the present embodiment, for example, q is a rotation angle of thejoint units 421 a to 421 f of the arm unit 420. An equation of motionregarding the operation space x is described by the following expression(2).

[Math. 2]

{umlaut over (x)}=Λ ⁻¹ f+c  (2)

Here, f represents a force acting on the operation space x. Furthermore,Λ⁻¹ is an operation space inertia inverse matrix, and c is calledoperation space bias acceleration, which are respectively expressed bythe following expressions (3) and (4).

[Math. 3]

Λ⁻¹ =JH ⁻¹ J ^(T)  (3)

c=JH ⁻¹(τ−b)+{dot over (J)}{dot over (q)}  (4)

Note that H represents a joint space inertia matrix, τ represents ajoint force corresponding to the joint value q (for example, thegenerated torque at the joint units 421 a to 421 f), and b representsgravity, a Coriolis force, and a centrifugal force.

In the generalized inverse dynamics, it is known that the motion purposeof the position and speed regarding the operation space x can beexpressed as an acceleration of the operation space x. At this time, thevirtual force f_(v) to act on the operation space x to realize anoperation space acceleration that is a target value given as the motionpurpose can be obtained by solving a kind of linear complementaryproblem (LCP) as in the expression (5) below according to the aboveexpression (1).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 4} \right\rbrack & \; \\{{{w + \overset{¨}{x}} = {{\Lambda^{- 1}f_{v}} + c}}{s.t.\left\{ \begin{matrix}{\left( {\left( {w_{i} < 0} \right)\bigcap\left( {f_{v_{i}} = U_{i}} \right)} \right)\bigcup} \\{\left( {\left( {w_{i} > 0} \right)\bigcap\left( {f_{v_{i}} = L_{i}} \right)} \right)\bigcup} \\\left( {\left( {w_{i} = 0} \right)\bigcap\left( {L_{i} < f_{v_{i}} < U_{i}} \right)} \right)\end{matrix} \right.}} & (5)\end{matrix}$

Here, L_(i) and U_(i) respectively represent a negative lower limitvalue (including −∞) of an i-th component of f_(v) and a positive upperlimit value (including +∞) of the i-th component of f_(v). The above LCPcan be solved using, for example, an iterative method, a pivot method, amethod applying robust acceleration control, or the like.

Note that the operation space inertia inverse matrix Λ⁻¹ and the biasacceleration c have a large calculation cost when calculated accordingto the expressions (3) and (4) that are defining expressions. Therefore,a method of calculating the processing of calculating the operationspace inertia inverse matrix Λ⁻¹ at a high speed by applying aquasi-dynamics operation (FWD) for obtaining a generalized acceleration(joint acceleration) from the generalized force (joint force τ) of themultilink structure has been proposed. Specifically, the operation spaceinertia inverse matrix Λ⁻¹ and the bias acceleration c can be obtainedfrom information regarding forces acting on the multilink structure (forexample, the joint units 421 a to 421 f of the arm unit 420), such asthe joint space q, the joint force τ, and the gravity g by using theforward dynamics operation FWD. The operation space inertia inversematrix Λ⁻¹ can be calculated with a calculation amount of O (N) withrespect to the number (N) of the joint units by applying the forwarddynamics operation FWD regarding the operation space.

Here, as a setting example of the motion purpose, a condition forachieving the target value (expressed by adding a superscript bar tosecond-order differentiation of x) of the operation space accelerationwith a virtual force f_(vi) equal to or smaller than an absolute valueF_(i) can be expressed by the following expression (6).

[Math. 5]

L _(i) =−F _(i),

U _(i) =F _(i),

{umlaut over (x)} _(i)={umlaut over ( x )}_(i)  (6)

Furthermore, as described above, the motion purpose regarding theposition and speed of the operation space x can be expressed as thetarget value of the operation space acceleration, and is specificallyexpressed by the following expression (7) (the target value of theposition and speed of the operation space x is expressed by x and addingthe superscript bar to first-order differentiation of x).

[Math. 6]

{umlaut over ( x )}_(i) =K _(p)( x _(i) −x _(i))+K _(v)({umlaut over ( x)}_(i) −{dot over (x)} _(i))  (7)

In addition, by use of a concept of decomposition operation space, themotion purpose regarding an operation space (momentum, Cartesianrelative coordinates, interlocking joint, or the like) expressed by alinear sum of other operation spaces. Note that it is necessary to givepriority to competing motion purposes. The above LCP can be solved foreach priority in ascending order from a low priority, and the virtualforce obtained by the LCP in the previous stage can be made to act as aknown external force of the LCP in the next stage.

(2-2-2. Real Force Calculation Processing)

In the real force calculation processing as the second stage of thegeneralized inverse dynamics, processing of replacing the virtual forcef_(v) obtained in the above (2-2-1. Virtual Force Determination Process)with real joint force and external force is performed. A condition forrealizing the generalized force τ_(v)=J_(v) ^(T)f_(v) by the virtualforce with a generated torque τ_(a) and an external force f_(e)generated in the joint unit is expressed by the following expression(8).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 7} \right\rbrack & \; \\{{\begin{bmatrix}J_{vu}^{T} \\J_{va}^{T}\end{bmatrix}\left( {f_{v} - {\Delta \; f_{v}}} \right)} = {{\begin{bmatrix}J_{eu}^{T} \\J_{ea}^{T}\end{bmatrix}f_{e}} + \begin{bmatrix}0 \\\tau_{a}\end{bmatrix}}} & (8)\end{matrix}$

Here, the suffix a represents a set of drive joint units (drive jointset), and the suffix u represents a set of non-drive joint units(non-drive joint set). In other words, the upper part of the aboveexpression (8) represents balance of the forces of the space (non-drivejoint space) by the non-drive joint units, and the lower part representsbalance of the forces of the space (drive joint space) by the drivejoint units. J_(vu) and J_(va) are respectively a non-drive jointcomponent and a drive joint component of the Jacobian regarding theoperation space where the virtual force f_(v) acts. J_(eu) and J_(ea)are a non-drive joint component and a drive joint component of theJacobian regarding the operation space where the external force f_(e)acts. Δf_(v) represents an unrealizable component with the real force,of the virtual force f_(v).

The upper part of the expression (8) is undefined. For example, f_(e)and Δf_(v) can be obtained by solving a quadratic programing problem(QP) as described in the following expression (9).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 8} \right\rbrack & \; \\{{{\min \frac{1}{2}ɛ^{T}Q_{1}ɛ} + {\frac{1}{2}\xi^{T}Q_{2}\xi}}{{{s.t.\mspace{14mu} U}\; \xi} \geq v}} & (9)\end{matrix}$

Here, ε is a difference between both sides of the upper part of theexpression (8), and represents an equation error of the expression (8).ξ is a connected vector of f_(e) and Δf_(v) and represents a variablevector. Q₁ and Q₂ are positive definite symmetric matrices thatrepresent weights at minimization. Furthermore, inequality constraint ofthe expression (9) is used to express the constraint condition regardingthe external force such as the vertical reaction force, friction cone,maximum value of the external force, or support polygon. For example,the inequality constraint regarding a rectangular support polygon isexpressed by the following expression (10).

[Math. 9]

|F _(x)|≤μ_(t) F _(z),

|F _(y)|≤μ_(t) F _(z),

F _(z)≥0,

|M _(x) |≤d _(t) F _(z),

|M _(y) |≤d _(x) F _(z),

|M _(x)|≤μ_(r) F _(z),  (10)

Here, z represents a normal direction of a contact surface, and x and yrepresent orthogonal two-tangent directions perpendicular to z. (F_(x),F_(y), F_(z)) and (M_(x), M_(y), M_(z)) represent an external force andan external force moment acting on a contact point. μ_(t) and μ_(r) arefriction coefficients regarding translation and rotation, respectively.(d_(x), d_(y)) represents the size of the support polygon.

From the above expressions (9) and (10), solutions f_(e) and Δf_(v) of aminimum norm or a minimum error are obtained. By substituting f_(e) andΔf_(v) obtained from the above expression (9) into the lower part of theabove expression (8), the joint force τ_(a) necessary for realizing themotion purpose can be obtained.

In a case of a system where a base is fixed and there is no non-drivejoint, all virtual forces can be replaced only with the joint force, andf_(e)=0 and Δf_(v)=0 can be set in the above expression (8). In thiscase, the following expression (11) can be obtained for the joint forceτ_(a) from the lower part of the above expression (8).

[Math. 10]

τ_(a) =J _(va) ^(T) f _(v)  (11)

The whole body coordination control using the generalized inversedynamics according to the present embodiment has been described. Bysequentially performing the virtual force calculation processing and thereal force calculation processing as described above, the joint forceτ_(a) for achieving a desired motion purpose can be obtained. In otherwords, conversely speaking, by reflecting the calculated joint forceτ_(a) in a theoretical model in the motion of the joint units 421 a to421 f, the joint units 421 a to 421 f are driven to achieve the desiredmotion purpose.

Note that, regarding the whole body coordination control using thegeneralized inverse dynamics described so far, in particular, details ofthe process of deriving the virtual force f_(v), the method of solvingthe LCP to obtain the virtual force f_(v), the solution of the QPproblem, and the like, reference can be made to Japanese PatentApplication Laid-Open Nos. 2009-95959 and 2010-188471, which are priorpatent applications filed by the present applicant, for example.

<2-3. Ideal Joint Control>

Next, the ideal joint control according to the present embodiment willbe described. The motion of each of the joint units 421 a to 421 f ismodeled by the equation of motion of the second-order lag system of thefollowing expression (12).

[Math. 11]

I _(a) {umlaut over (q)}=τ _(a)+τ_(e) −v _(a) {dot over (q)}  (12)

Here, I_(a) represents moment of inertia (inertia) at the joint unit,τ_(a) represents the generated torque of the joint units 421 a to 421 f,τ _(e) represents external torque acting on each of the joint units 421a to 421 f from the outside, and ν_(e) represents a viscous dragcoefficient in each of the joint units 421 a to 421 f. The aboveexpression (12) can also be said to be a theoretical model thatrepresents the motion of the actuators in the joint units 421 a to 421f.

τ_(a) that is the real force to act on each of the joint units 421 a to421 f for realizing the motion purpose can be calculated using themotion purpose and the constraint condition by the operation using thegeneralized inverse dynamics described in <2-2. Generalized InverseDynamics> above. Therefore, ideally, by applying each calculated τ_(a)to the above expression (12), a response according to the theoreticalmodel illustrated in the above expression (12) is realized, in otherwords, the desired motion purpose should be achieved.

However, in practice, errors (modeling errors) may occur between themotions of the joint units 421 a to 421 f and the theoretical model asillustrated in the above expression (12), due to the influence ofvarious types of disturbance. The modeling errors can be roughlyclassified into those due to mass property such as weight, center ofgravity, inertia tensor of the multilink structure, and those due tofriction, inertia, and the like inside joint units 421 a to 421 f. Amongthem, the modeling errors due to the former mass property can berelatively easily reduced at the time of constructing the theoreticalmodel by improving the accuracy of computer aided design (CAD) data andapplying an identification method.

Meanwhile, the modeling errors due to the latter friction, inertia, andthe like inside the joint units 421 a to 421 f are caused by phenomenathat are difficult to model, such as friction in a reduction gear 426 ofthe joint units 421 a to 421 f, for example, and a modeling error thatcannot be ignored may remain during model construction. Furthermore,there is a possibility that an error occurs between the values of theinertia I_(a) and the viscous drag coefficient ν_(e) in the aboveexpression (12) and the values in the actual joint units 421 a to 421 f.These errors that are difficult to model can become the disturbance inthe drive control of the joint units 421 a to 421 f. Therefore, inpractice, the motions of the joint units 421 a to 421 f may not respondaccording to the theoretical model illustrated in the above expression(12), due to the influence of such disturbance. Therefore, even when thereal force τ_(a), which is a joint force calculated by the generalizedinverse dynamics, is applied, there may be a case where the motionpurpose that is the control target is not achieved. In the presentembodiment, correcting the responses of the joint units 421 a to 421 fso as to perform ideal responses according to the theoretical modelillustrated in the above expression (12), by adding an active controlsystem to each of the joint units 421 a to 421 f, is considered.Specifically, in the present embodiment, not only performing frictioncompensation type torque control using the torque sensors 428 and 428 aof the joint units 421 a to 421 f but also performing an ideal responseaccording to the theoretical values up to the inertia I_(a) and theviscous drag coefficient ν_(a) to the required generated torque τ_(a)and external torque τ_(e).

In the present embodiment, control of the driving of the joint units 421a to 421 f of the support arm device 400 to perform ideal responses asdescribed in the above expression (12) is called ideal joint control.Here, in the following description, an actuator controlled to be drivenby the ideal joint control is also referred to as a virtualized actuator(VA) because of performing an ideal response. Hereinafter, the idealjoint control according to the present embodiment will be described withreference to FIG. 4.

FIG. 4 is an explanatory diagram for describing the ideal joint controlaccording to an embodiment of the present disclosure. Note that FIG. 4schematically illustrates a conceptual arithmetic unit that performsvarious operations regarding the ideal joint control in blocks.

Here, a response of an actuator 610 according to the theoretical modelexpressed by the above expression (12) is nothing less than achievementof the rotation angular acceleration on the left side when the rightside of the expression (12) is given. Furthermore, as illustrated in theabove expression (12), the theoretical model includes an external torqueterm τ_(e) acting on the actuator 610. In the present embodiment, theexternal torque term τ_(e) is measured by a torque sensor 614 in orderto perform the ideal joint control. Furthermore, a disturbance observer620 is applied to calculate a disturbance estimation value I_(d) that isan estimation value of a torque due to disturbance on the basis of arotation angle q of the actuator 610 measured by an encoder 613.

A block 631 represents an arithmetic unit that performs an operationaccording to an ideal joint model of the joint units 421 a to 421 fillustrated in the above expression (12). The block 631 can output arotation angular acceleration target value (a second derivative of arotation angle target value q^(ref) described on the left side of theabove expression (12), using the generated torque τ_(a), the externaltorque τ_(e), and the rotation angular speed (first-orderdifferentiation of the rotation angle q) as inputs.

In the present embodiment, the generated torque τ_(a) calculated by themethod described in <2-2. Generalized inverse dynamics> above and theexternal torque τ_(e) measured by the torque sensor 614 are input to theblock 631. Meanwhile, when the rotation angle q measured by the encoder613 is input to a block 632 representing an arithmetic unit thatperforms a differential operation, the rotation angular speed (thefirst-order differentiation of the rotation angle q) is calculated. Whenthe rotation angular speed calculated in the block 632 is input to theblock 631 in addition to the generated torque τ_(a) and the externaltorque τ_(er) the rotation angular acceleration target value iscalculated by the block 631. The calculated rotation angularacceleration target value is input to a block 633.

The block 633 represents an arithmetic unit that calculates a torquegenerated in the actuator 610 on the basis of the rotation angularacceleration of the actuator 610. In the present embodiment,specifically, the block 633 can obtain a torque target value τ^(ref) bymultiplying the rotation angular acceleration target value by nominalinertia J_(n) in the actuator 610. In the ideal response, the desiredmotion purpose should be achieved by causing the actuator 610 togenerate the torque target value τ^(ref). However, as described above,there is a case where the influence of the disturbance or the likeoccurs in the actual response. Therefore, in the present embodiment, thedisturbance observer 620 calculates the disturbance estimation valueτ_(d) and corrects the torque target value τ^(ref) using the disturbanceestimation value τ_(d).

A configuration of the disturbance observer 620 will be described. Asillustrated in FIG. 4, the disturbance observer 620 calculates thedisturbance estimation value τ_(d) on the basis of the torque commandvalue T and the rotation angular speed output from the rotation angle qmeasured by the encoder 613. Here, the torque command value τ is atorque value to be finally generated in the actuator 610 after theinfluence of disturbance is corrected. For example, win a case where thedisturbance estimation value τ_(d) is not calculated, the torque commandvalue τ becomes the torque target value τ^(ref).

The disturbance observer 620 includes a block 634 and a block 635. Theblock 634 represents an arithmetic unit that calculates a torquegenerated in the actuator 610 on the basis of the rotation angular speedof the actuator 610. In the present embodiment, specifically, therotation angular speed calculated by the block 632 from the rotationangle q measured by the encoder 613 is input to the block 634. The block634 obtains the rotation angular acceleration by performing an operationrepresented by a transfer function J_(n)s, in other words, bydifferentiating the rotation angular speed, and further multiplies thecalculated rotation angular acceleration by the nominal inertia J_(n),thereby calculating an estimation value of the torque actually acting onthe actuator 610 (torque estimation value).

In the disturbance observer 620, the difference between the torqueestimation value and the torque command value τ is obtained, whereby thedisturbance estimation value τ_(d), which is the value of the torque dueto the disturbance, is estimated. Specifically, the disturbanceestimation value τ_(d) may be a difference between the torque commandvalue τ in the control of the preceding cycle and the torque estimationvalue in the current control. Since the torque estimation valuecalculated by the block 634 is based on the actual measurement value andthe torque command value τ calculated by the block 633 is based on theideal theoretical model of the joint units 421 a to 421 f illustrated inthe block 631, the influence of the disturbance, which is not consideredin the theoretical model, can be estimated by taking the differencebetween the torque estimation value and the torque command value τ.

Furthermore, the disturbance observer 620 is provided with a low passfilter (LPF) illustrated in a block 635 to prevent system divergence.The block 635 outputs only a low frequency component to the input valueby performing an operation represented by a transfer function g/(s+g) tostabilize the system. In the present embodiment, the difference valuebetween the torque estimation value and the torque command value τ^(ref)calculated by the block 634 is input to the block 635, and a lowfrequency component of the difference value is calculated as thedisturbance estimation value τ_(d).

In the present embodiment, feedforward control to add the disturbanceestimation value τ_(d) calculated by the disturbance observer 620 to thetorque target value τ^(ref) is performed, whereby the torque commandvalue T that is the torque value to be finally generated in the actuator610 is calculated. Then, the actuator 610 is driven on the basis of thetorque command value τ. Specifically, the torque command value τ isconverted into a corresponding current value (current command value),and the current command value is applied to a motor 611, so that theactuator 610 is driven.

As described above, with the configuration described with reference toFIG. 4, the response of the actuator 610 can be made to follow thetarget value even in a case where there is a disturbance component suchas friction in the drive control of the joint units 421 a to 421 faccording to the present embodiment. Furthermore, with regard to thedrive control of the joint units 421 a to 421 f, an ideal responseaccording to the inertia I_(a) and the viscous drag coefficient ν_(a)assumed by the theoretical model can be made.

Note that, for details of the above-described ideal joint control,Japanese Patent Application Laid-Open No. 2009-269102, which is a priorpatent application filed by the present applicant, can be referred to,for example.

The generalized inverse dynamics used in the present embodiment has beendescribed, and the ideal joint control according to the presentembodiment has been described with reference to FIG. 4. As describedabove, in the present embodiment, the whole body coordination control,in which the drive parameters of the joint units 421 a to 421 f (forexample, the generated torque values of the joint units 421 a to 421 f)for achieving the motion purpose of the arm unit 420 are calculated inconsideration of the constraint condition, is performed using thegeneralized inverse dynamics. Furthermore, as described with referenceto FIG. 4, in the present embodiment, the ideal joint control thatrealizes the ideal response based on the theoretical model in the drivecontrol of the joint units 421 a to 421 f by performing correction ofthe generated torque value, which has been calculated in the whole bodycoordination control using the generalized inverse dynamics, inconsideration of the influence of the disturbance, is performed.Therefore, in the present embodiment, highly accurate drive control thatachieves the motion purpose becomes possible with regard to the drivingof the arm unit 420.

<2-4. Configuration of Arm Control System>

Next, a configuration of an arm control system according to the presentembodiment, in which the whole body coordination control and the idealjoint control described in <2-2. Generalized Inverse Dynamics> and <2-3.Ideal Joint Control> above are applied to drive control of an arm devicewill be described.

A configuration example of an arm control system according to anembodiment of the present disclosure will be described with reference toFIG. 5. FIG. 5 is a functional block diagram illustrating aconfiguration example of an arm control system according to anembodiment of the present disclosure. Note that, in the arm controlsystem illustrated in FIG. 5, a configuration related to drive controlof an arm unit of an arm device will be mainly illustrated.

Referring to FIG. 5, an arm control system 1 according to an embodimentof the present disclosure includes an arm device 10, a control device20, and a display device 30. In the present embodiment, the controldevice 20 performs various operations in the whole body coordinationcontrol described in <2-2. Generalized Inverse Dynamics> and the idealjoint control described in <2-3. Ideal Joint Control> above, and drivingof the arm unit of the arm device 10 is controlled on the basis of anoperation result. Furthermore, the arm unit of the arm device 10 isprovided with an imaging unit 140 described below, and an image capturedby the imaging unit 140 is displayed on a display screen of the displaydevice 30. Hereinafter, configurations of the arm device 10, the controldevice 20, and the display device 30 will be described in detail.

The arm device 10 includes the arm unit that is a multilink structureincluding a plurality of joint units and a plurality of links, anddrives the arm unit within a movable range to control the position andposture of a distal end unit provided at a distal end of the arm unit.The arm device 10 corresponds to the support arm device 400 illustratedin FIG. 3.

Referring to FIG. 5, the arm device 10 includes an arm control unit 110and an arm unit 120. Furthermore, the arm unit 120 includes a joint unit130 and the imaging unit 140.

The arm control unit 110 integrally controls the arm device 10 andcontrols driving of the arm unit 120. The arm control unit 110corresponds to the control unit (not illustrated in FIG. 3) describedwith reference to FIG. 3. Specifically, the arm control unit 110includes a drive control unit 111. Driving of the joint unit 130 iscontrolled by the control of the drive control unit 111, so that thedriving of the arm unit 120 is controlled. More specifically, the drivecontrol unit 111 controls a current amount to be supplied to a motor inan actuator of the joint unit 130 to control the number of rotations ofthe motor, thereby controlling a rotation angle and generated torque inthe joint unit 130. However, as described above, the drive control ofthe arm unit 120 by the drive control unit 111 is performed on the basisof the operation result in the control device 20. Therefore, the currentamount to be supplied to the motor in the actuator of the joint unit130, which is controlled by the drive control unit 111, is a currentamount determined on the basis of the operation result in the controldevice 20.

The arm unit 120 is a multilink structure including a plurality ofjoints and a plurality of links, and driving of the arm unit 120 iscontrolled by the control of the arm control unit 110. The arm unit 120corresponds to the arm unit 420 illustrated in FIG. 3. The arm unit 120includes the joint unit 130 and the imaging unit 140. Note that, sincefunctions and structures of the plurality of joint units included in thearm unit 120 are similar to one another, FIG. 5 illustrates aconfiguration of one joint unit 130 as a representative of the pluralityof joint units.

The joint unit 130 rotatably connects the links with each other in thearm unit 120, and drives the arm unit 120 as rotational driving of thejoint unit 130 is controlled by the control of the arm control unit 110.The joint unit 130 corresponds to the joint units 421 a to 421 fillustrated in FIG. 3. Furthermore, the joint unit 130 includes anactuator.

The joint unit 130 includes a joint drive unit 131 and a joint statedetection unit 132.

The joint drive unit 131 is a drive mechanism in the actuator of thejoint unit 130, and the joint unit 130 is rotationally driven as thejoint drive unit 131 is driven. The driving of the joint drive unit 131is controlled by the drive control unit 111. For example, the jointdrive unit 131 is a configuration corresponding to the motor and a motordriver, and the joint drive unit 131 being driven corresponds to themotor driver driving the motor with the current amount according to acommand from the drive control unit 111.

The joint state detection unit 132 detects a state of the joint unit130. Here, the state of the joint unit 130 may mean a state of motion ofthe joint unit 130. For example, the state of the joint unit 130includes information of the rotation angle, rotation angular speed,rotation angular acceleration, generated torque of the joint unit 130,and the like. In the present embodiment, the joint state detection unit132 has a rotation angle detection unit 133 that detects the rotationangle of the joint unit 130 and a torque detection unit 134 that detectsthe generated torque and the external torque of the joint unit 130. Notethat the rotation angle detection unit 133 and the torque detection unit134 correspond to an encoder and a torque sensor of the actuator,respectively. The joint state detection unit 132 transmits the detectedstate of the joint unit 130 to the control device 20.

The imaging unit 140 is an example of the distal end unit provided atthe distal end of the arm unit 120, and acquires an image of a capturetarget. The imaging unit 140 corresponds to the imaging unit 423illustrated in FIG. 3. Specifically, the imaging unit 140 is a camera orthe like that can capture the capture target in the form of a movingimage or a still image. More specifically, the imaging unit 140 includesa plurality of light receiving elements arranged in a two dimensionalmanner, and can obtain an image signal representing an image of thecapture target by photoelectric conversion in the light receivingelements. The imaging unit 140 transmits the acquired image signal tothe display device 30.

Note that, as in the case of the support arm device 400 illustrated inFIG. 3, the imaging unit 423 is provided at the distal end of the armunit 420, the imaging unit 140 is actually provided at the distal end ofthe arm unit 120 in the robot arm device 10. FIG. 5 illustrates a statein which the imaging unit 140 is provided at a distal end of a finallink via the plurality of joint units 130 and the plurality of links byschematically illustrating a link between the joint unit 130 and theimaging unit 140.

Note that, in the present embodiment, various medical instruments can beconnected to the distal end of the arm unit 120 as the distal end unit.Examples of the medical instruments include various treatmentinstruments such as a scalpel and forceps, and various units used intreatment, such as a unit of various detection devices such as probes ofan ultrasonic examination device. Furthermore, in the presentembodiment, the imaging unit 140 illustrated in FIG. 5 or a unit havingan imaging function such as an endoscope or a microscope may also beincluded in the medical instruments. Thus, the arm device 10 accordingto the present embodiment can be said to be a medical arm deviceprovided with medical instruments. Similarly, the arm control system 1according to the present embodiment can be said to be a medical armcontrol system. Note that the arm device 10 illustrated in FIG. 5 canalso be said to be a scope holding arm device provided with a unithaving an imaging function as the distal end unit. Furthermore, a stereocamera having two imaging units (camera units) may be provided at thedistal end of the arm unit 120, and may capture an imaging target to bedisplayed as a 3D image.

The function and configuration of the arm device 10 have been describedabove. Next, a function and a configuration of the control device 20will be described. Referring to FIG. 5, the control device 20 includesan input unit 210, a storage unit 220, and a control unit 230.

The control unit 230 integrally controls the control device 20 andperforms various operations for controlling the driving of the arm unit120 in the arm device 10. Specifically, to control the driving of thearm unit 120 of the arm device 10, the control unit 230 performs variousoperations in the whole body coordination control and the ideal jointcontrol. Hereinafter, the function and configuration of the control unit230 will be described in detail. The whole body coordination control andthe ideal joint control have been already described in <2-2. GeneralizedInverse Dynamics> and <2-3. Ideal Joint Control> above, and thusdetailed description is omitted here.

The control unit 230 includes a whole body coordination control unit 240and an ideal joint control unit 250.

The whole body coordination control unit 240 performs various operationsregarding the whole body coordination control using the generalizedinverse dynamics. In the present embodiment, the whole body coordinationcontrol unit 240 acquires a state (arm state) of the arm unit 120 on thebasis of the state of the joint unit 130 detected by the joint statedetection unit 132. Furthermore, the whole body coordination controlunit 240 calculates a control value for the whole body coordinationcontrol of the arm unit 120 in an operation space, using the generalizedinverse dynamics, on the basis of the arm state, and a motion purposeand a constraint condition of the arm unit 120. Note that the operationspace is a space for describing the relationship between the forceacting on the arm unit 120 and the acceleration generated in the armunit 120, for example.

The whole body coordination control unit 240 includes an arm stateacquisition unit 241, an arithmetic condition setting unit 242, avirtual force calculation unit 243, and a real force calculation unit244.

The arm state acquisition unit 241 acquires the state (arm state) of thearm unit 120 on the basis of the state of the joint unit 130 detected bythe joint state detection unit 132. Here, the arm state may mean thestate of motion of the arm unit 120. For example, the arm state includesinformation such as the position, speed, acceleration, and force of thearm unit 120. As described above, the joint state detection unit 132acquires, as the state of the joint unit 130, the information of therotation angle, rotation angular speed, rotation angular acceleration,generated torque, and the like in each joint unit 130. Furthermore,although to be described below, the storage unit 220 stores varioustypes of information to be processed by the control device 20. In thepresent embodiment, the storage unit 220 may store various types ofinformation (arm information) regarding the arm unit 120, for example,the number of joint units 130 and links configuring the arm unit 120,connection states between the links and the joint units 130, and lengthsof the links, and the like. The arm state acquisition unit 241 canacquire the arm information from the storage unit 220. Therefore, thearm state acquisition unit 241 can acquire, as the arm state,information such as the positions (coordinates) in the space of theplurality of joint units 130, the plurality of links, and the imagingunit 140 (in other words, the shape of the arm unit 120 and the positionand posture of the imaging unit 140), and the forces acting on the jointunits 130, the links, and the imaging unit 140, on the basis of thestate and the arm information of the joint units 130. The arm stateacquisition unit 241 transmits the acquired arm information to thearithmetic condition setting unit 242.

The arithmetic condition setting unit 242 sets operation conditions inan operation regarding the whole body coordination control using thegeneralized inverse dynamics. Here, the operation condition may be a themotion purpose and a constraint condition. The motion purpose may bevarious types of information regarding the motion of the arm unit 120.Specifically, the motion purpose may be target values of the positionand posture (coordinates), speed, acceleration, force, and the like ofthe imaging unit 140, or target values of the positions (coordinates),speeds, accelerations, forces, and the like of the plurality of jointunits 130 and the plurality of links of the arm unit 120. Furthermore,the constraint condition may be various types of information thatrestricts (restrains) the motion of the arm unit 120. Specifically, theconstraint condition may be coordinates of a region where eachconfiguration component of the arm unit cannot move, an unmovable speed,a value of acceleration, a value of an ungenerable force, and the like.Furthermore, restriction ranges of various physical quantities under theconstraint condition may be set according to inability to structurallyrealizing the arm unit 120 or may be appropriately set by the user.Furthermore, the arithmetic condition setting unit 242 includes aphysical model for the structure of the arm unit 120 (in which, forexample, the number and lengths of the links configuring the arm unit120, the connection states of the links via the joint units 130, themovable ranges of the joint units 130, and the like are modeled), andmay set a motion condition and the constraint condition by generating acontrol model in which the desired motion condition and constraintcondition are reflected in the physical model.

In the present embodiment, appropriate setting of the motion purpose andthe constraint condition enables the arm unit 120 to perform a desiredoperation. For example, not only can the imaging unit 140 be moved to atarget position by setting a target value of the position of the imagingunit 140 as the motion purpose but also the arm unit 120 can be drivenby providing a constraint of movement by the constraint condition toprevent the arm unit 120 from intruding into a predetermined region inthe space.

A specific example of the motion purpose includes, for example, a pivotoperation, which is a turning operation with an axis of a cone servingas a pivot axis, in which the imaging unit 140 moves in a conicalsurface setting an operation site as a top in a state where the capturedirection of the imaging unit 140 is fixed to the operation site.Furthermore, in the pivot operation, the turning operation may beperformed in a state where the distance between the imaging unit 140 anda point corresponding to the top of the cone is kept constant. Byperforming such a pivot operation, an observation site can be observedfrom an equal distance and at different angles, whereby the convenienceof the user who performs surgery can be improved.

Furthermore, as another specific example, the motion purpose may becontent to control the generated torque in each joint unit 130.Specifically, the motion purpose may be a power assist operation tocontrol the state of the joint unit 130 to cancel the gravity acting onthe arm unit 120, and further control the state of the joint unit 130 tosupport the movement of the arm unit 120 in a direction of a forceprovided from the outside. More specifically, in the power assistoperation, the driving of each joint unit 130 is controlled to causeeach joint unit 130 to generate a generated torque that cancels theexternal torque due to the gravity in each joint unit 130 of the armunit 120, whereby the position and posture of the arm unit 120 are heldin a predetermined state. In a case where an external torque is furtheradded from the outside (for example, from the user) in theaforementioned state, the driving of each joint unit 130 is controlledto cause each joint unit 130 to generate a generated torque in the samedirection as the added external torque. By performing such a powerassist operation, the user can move the arm unit 120 with a smallerforce in a case where the user manually moves the arm unit 120.Therefore, a feeling as if the user moved the arm unit 120 underweightlessness can be provided to the user. Furthermore, theabove-described pivot operation and the power assist operation can becombined.

Here, in the present embodiment, the motion purpose may mean anoperation (motion) of the arm unit 120 realized by the whole bodycoordination control or may mean an instantaneous motion purpose in theoperation (in other words, a target value in the motion purpose). Forexample, in the above-described pivot operation, the imaging unit 140performing the pivot operation itself is the motion purpose. In the actof performing the pivot operation, values of the position, speed, andthe like of the imaging unit 140 in a conical surface in the pivotoperation are set as the instantaneous motion purpose (the target valuesin the motion purpose). Furthermore, in the above-described power assistoperation, for example, performing the power assist operation to supportthe movement of the arm unit 120 in the direction of the force appliedfrom the outside itself is the motion purpose. In the act of performingthe power assist operation, the value of the generated torque in thesame direction as the external torque applied to each joint unit 130 isset as the instantaneous motion purpose (the target value in the motionpurpose). The motion purpose in the present embodiment is a conceptincluding both the instantaneous motion purpose (for example, the targetvalues of the positions, speeds, forces, and the like of theconfiguration members of the arm unit 120 at a certain time) and theoperations of the configuration members of the arm unit 120 realizedover time as a result of the instantaneous motion purpose having beencontinuously achieved. The instantaneous motion purpose is set each timein each step in an operation for the whole body coordination control inthe whole body coordination control unit 240, and the operation isrepeatedly performed, so that the desired motion purpose is finallyachieved.

Note that, in the present embodiment, the viscous drag coefficient in arotation motion of each joint unit 130 may be appropriately set when themotion purpose is set. As described above, the joint unit 130 accordingto the present embodiment is configured to be able to appropriatelyadjust the viscous drag coefficient in the rotation motion of theactuator. Therefore, by setting the viscous drag coefficient in therotation motion of each joint unit 130 when setting the motion purpose,an easily rotatable state or a less easily rotatable state can berealized for the force applied from the outside, for example. Forexample, in the above-descried power assist operation, when the viscousdrag coefficient in the joint unit 130 is set to be small, a forcerequired by the user to move the arm unit 120 can be made small, and aweightless feeling provided to the user can be promoted. As describedabove, the viscous drag coefficient in the rotation motion of each jointunit 130 may be appropriately set according to the content of the motionpurpose.

Here, in the present embodiment, as will be described below, the storageunit 220 may store parameters regarding the operation conditions such asthe motion purpose and the constraint condition used in the operationregarding the whole body coordination control. The arithmetic conditionsetting unit 242 can set the constraint condition stored in the storageunit 220 as the constraint condition used for the operation of the wholebody coordination control.

Furthermore, in the present embodiment, the arithmetic condition settingunit 242 can set the motion purpose by a plurality of methods. Forexample, the arithmetic condition setting unit 242 may set the motionpurpose on the basis of the arm state transmitted from the arm stateacquisition unit 241. As described above, the arm state includesinformation of the position of the arm unit 120 and information of theforce acting on the arm unit 120. Therefore, for example, in a casewhere the user is trying to manually move the arm unit 120, informationregarding how the user is moving the arm unit 120 is also acquired bythe arm state acquisition unit 241 as the arm state. Therefore, thearithmetic condition setting unit 242 can set the position, speed,force, and the like to/at/with which the user has moved the arm unit120, as the instantaneous motion purpose, on the basis of the acquiredarm state. By thus setting the motion purpose, the driving of the armunit 120 is controlled to follow and support the movement of the armunit 120 by the user.

Furthermore, for example, the arithmetic condition setting unit 242 mayset the motion purpose on the basis of an instruction input from theinput unit 210 by the user. Although to be described below, the inputunit 210 is an input interface for the user to input information,commands, and the like regarding the drive control of the arm device 10,to the control device 20. In the present embodiment, the motion purposemay be set on the basis of an operation input from the input unit 210 bythe user. Specifically, the input unit 210 has, for example, operationmeans operated by the user, such as a lever and a pedal. The positions,speeds, and the like of the configuration members of the arm unit 120may be set as the instantaneous motion purpose by the arithmeticcondition setting unit 242 in response to an operation of the lever,pedal, or the like.

Moreover, for example, the arithmetic condition setting unit 242 may setthe motion purpose stored in the storage unit 220 as the motion purposeused for the operation of the whole body coordination control. Forexample, in the case of the motion purpose that the imaging unit 140stands still at a predetermined point in the space, coordinates of thepredetermined point can be set in advance as the motion purpose.Furthermore, for example, in the case of the motion purpose that theimaging unit 140 moves on a predetermined trajectory in the space,coordinates of each point representing the predetermined trajectory canbe set in advance as the motion purpose. As described above, in a casewhere the motion purpose can be set in advance, the motion purpose maybe stored in the storage unit 220 in advance. Furthermore, in the caseof the above-described pivot operation, for example, the motion purposeis limited to a motion purpose setting the position, speed, and the likein the conical surface as the target values. In the case of the powerassist operation, the motion purpose is limited to a motion purposesetting the force as the target value. In the case where the motionpurpose such as the pivot operation or the power assist operation is setin advance in this way, information regarding ranges, types and the likeof the target values settable as the instantaneous motion purpose insuch a motion purpose may be stored in the storage unit 220. Thearithmetic condition setting unit 242 can also set the various types ofinformation regarding such a motion purpose as the motion purpose.

Note that by which method the arithmetic condition setting unit 242 setsthe motion purpose may be able to be appropriately set by the useraccording to the application of the arm device 10 or the like.Furthermore, the arithmetic condition setting unit 242 may set themotion purpose and the constraint condition by appropriately combiningthe above-described methods. Note that a priority of the motion purposemay be set in the constraint condition stored in the storage unit 220,or in a case where is a plurality of motion purposes different from oneanother, the arithmetic condition setting unit 242 may set the motionpurpose according to the priority of the constraint condition. Thearithmetic condition setting unit 242 transmits the arm state and theset motion purpose and constraint condition to the virtual forcecalculation unit 243.

The virtual force calculation unit 243 calculates a virtual force in theoperation regarding the whole body coordination control using thegeneralized inverse dynamics. The processing of calculating the virtualforce performed by the virtual force calculation unit 243 may be theseries of processing described in, for example, <2-2-1. Virtual ForceCalculation Processing> above. The virtual force calculation unit 243transmits the calculated virtual force f_(v) to the real forcecalculation unit 244.

The real force calculation unit 244 calculates a real force in theoperation regarding the whole body coordination control using thegeneralized inverse dynamics. The processing of calculating the realforce performed by the real force calculation unit 244 may be the seriesof processing described in, for example, <2-2-2. Real Force CalculationProcessing> above. The real force calculation unit 244 transmits thecalculated real force (generated torque) τ_(a) to the ideal jointcontrol unit 250. Note that, in the present embodiment, the generatedtorque τ_(a) calculated by the real force calculation unit 244 is alsoreferred to as a control value or a control torque value in the sense ofa control value of the joint unit 130 in the whole body coordinationcontrol.

The ideal joint control unit 250 performs various operations regardingthe ideal joint control using the generalized inverse dynamics. In thepresent embodiment, the ideal joint control unit 250 correct theinfluence of disturbance for the generated torque τ_(a) calculated bythe real force calculation unit 244 to calculate a torque command valueτ realizing an ideal response of the arm unit 120. Note that thearithmetic processing performed by the ideal joint control unit 250corresponds to the series of processing described in <2-3. Ideal JointControl> above.

The ideal joint control unit 250 includes a disturbance estimation unit251 and a command value calculation unit 252.

The disturbance estimation unit 251 calculates a disturbance estimationvalue τ_(d) on the basis of the torque command value τ and the rotationangular speed calculated from the rotation angle q detected by therotation angle detection unit 133. Note that the torque command value τmentioned here is a command value that represents the generated torquein the arm unit 120 to be finally transmitted to the arm device 10.Thus, the disturbance estimation unit 251 has a function correspondingto the disturbance observer 620 illustrated in FIG. 4.

The command value calculation unit 252 calculates the torque commandvalue τ that is a command value representing the torque to be generatedin the arm unit 120 and finally transmitted to the arm device 10, usingthe disturbance estimation value τ_(d) calculated by the disturbanceestimation unit 251. Specifically, the command value calculation unit252 adds the disturbance estimation value I_(d) calculated by thedisturbance estimation unit 251 to τ^(ref) calculated from the idealmodel of the joint unit 130 described in the above expression (12) tocalculate the torque command value τ. For example, win a case where thedisturbance estimation value τ_(d) is not calculated, the torque commandvalue τ becomes the torque target value τ^(ref). Thus, the function ofthe command value calculation unit 252 corresponds to the function otherthan the disturbance observer 620 illustrated in FIG. 4.

As described above, in the ideal joint control unit 250, the informationis repeatedly exchanged between the disturbance estimation unit 251 andthe command value calculation unit 252, so that the series of processingdescribed with reference to FIG. 4 is performed. The ideal joint controlunit 250 transmits the calculated torque command value τ to the drivecontrol unit 111 of the arm device 10. The drive control unit 111performs control to supply the current amount corresponding to thetransmitted torque command value τ to the motor in the actuator of thejoint unit 130, thereby controlling the number of rotations of the motorand controlling the rotation angle and the generated torque in the jointunit 130.

In the arm control system 1 according to the present embodiment, thedrive control of the arm unit 120 in the arm device 10 is continuouslyperformed during work using the arm unit 120, so the above-describedprocessing in the arm device 10 and the control device 20 is repeatedlyperformed. In other words, the state of the joint unit 130 is detectedby the joint state detection unit 132 of the arm device 10 andtransmitted to the control device 20. The control device 20 performsvarious operations regarding the whole body coordination control and theideal joint control for controlling the driving of the arm unit 120 onthe basis of the state of the joint unit 130, and the motion purpose andthe constraint condition, and transmits the torque command value τ asthe operation result to the arm device 10. The arm device 10 controlsthe driving of the arm unit 120 on the basis of the torque command valueτ, and the state of the joint unit 130 during or after the driving isdetected by the joint state detection unit 132 again.

Description about other configurations included in the control device 20will be continued.

The input unit 210 is an input interface for the user to inputinformation, commands, and the like regarding the drive control of thearm device 10 to the control device 20. In the present embodiment, thedriving of the arm unit 120 of the arm device 10 may be controlled onthe basis of the operation input from the input unit 210 by the user,and the position and posture of the imaging unit 140 may be controlled.Specifically, as described above, instruction information regarding theinstruction of the driving of the arm input from the input unit 210 bythe user is input to the arithmetic condition setting unit 242, so thatthe arithmetic condition setting unit 242 may set the motion purpose inthe whole body coordination control on the basis of the instructioninformation. The whole body coordination control is performed using themotion purpose based on the instruction information input by the user asdescribed above, so that the driving of the arm unit 120 according tothe operation input of the user is realized.

Specifically, the input unit 210 includes operation means operated bythe user, such as a mouse, a keyboard, a touch panel, a button, aswitch, a lever, and a pedal, for example. For example, in a case wherethe input unit 210 has a pedal, the user can control the driving of thearm unit 120 by operating the pedal with the foot. Therefore, even in acase where the user is performing treatment using both hands on theoperation site of the patient, the user can adjust the position andposture of the imaging unit 140, in other words, the user can adjust acapture position and a capture angle of the operation site, by theoperation of the pedal with the foot.

The storage unit 220 stores various types of information processed bythe control device 20. In the present embodiment, the storage unit 220can store various parameters used in the operation regarding the wholebody coordination control and the ideal joint control performed by thecontrol unit 230. For example, the storage unit 220 may store the motionpurpose and the constraint condition used in the operation regarding thewhole body coordination control by the whole body coordination controlunit 240. The motion purpose stored in the storage unit 220 may be, asdescribed above, a motion purpose that can be set in advance, such as,for example, the imaging unit 140 standing still at a predeterminedpoint in the space. Furthermore, the constraint conditions may be set inadvance by the user and stored in the storage unit 220 according to ageometric configuration of the arm unit 120, the application of therobot arm device 10, and the like. Furthermore, the storage unit 220 mayalso store various types of information regarding the arm unit 120 usedwhen the arm state acquisition unit 241 acquires the arm state.Moreover, the storage unit 220 may store the operation result, variousnumerical values, and the like calculated in the operation process inthe operation regarding the whole body coordination control and theideal joint control by the control unit 230. As described above, thestorage unit 220 may store any parameters regarding the various types ofprocessing performed by the control unit 230, and the control unit 230can performs various types of processing while mutually exchanginginformation with the storage unit 220.

The function and configuration of the control device 20 have beendescribed above. Note that the control device 20 according to thepresent embodiment can be configured by, for example, variousinformation processing devices (arithmetic processing devices) such as apersonal computer (PC) and a server. Next, a function and aconfiguration of the display device 30 will be described.

The display device 30 displays the information on the display screen invarious formats such as texts and images to visually notify the user ofvarious types of information. In the present embodiment, the displaydevice 30 displays the image captured by the imaging unit 140 of the armdevice 10 on the display screen. Specifically, the display device 30 hasfunctions and configurations of an image signal processing unit (notillustrated) that applies various types of image processing to an imagesignal acquired by the imaging unit 140, a display control unit (notillustrated) that performs control to display an image based on theprocessed image signal on the display screen, and the like. Note thatthe display device 30 may have various functions and configurations thata display device generally has, in addition to the above-describedfunctions and configurations. The display device 30 corresponds to thedisplay device 5041 illustrated in FIG. 1.

The functions and configurations of the arm device 10, the controldevice 20, and the display device 30 according to the present embodimenthave been described above with reference to FIG. 5. Each of theabove-described constituent elements may be configured usinggeneral-purpose members or circuit, or may be configured by hardwarespecialized for the function of each constituent element. Furthermore,all the functions of the configuration elements may be performed by aCPU or the like. Therefore, the configuration to be used can be changedas appropriate according to the technical level of the time of carryingout the present embodiment.

As described above, according to the present embodiment, the arm unit120 that is the multilink structure in the arm device 10 has at leastsix degrees or more of freedom, and the driving of each of the pluralityof joint units 130 configuring the arm unit 120 is controlled by thedrive control unit 111. Then, a medical instrument is provided at thedistal end of the arm unit 120. The driving of each of the joint units130 is controlled as described above, so that the drive control of thearm unit 120 with a higher degree of freedom is realized, and themedical arm device 10 with higher operability for the user is realized.

More specifically, according to the present embodiment, the joint statedetection unit 132 detects the state of the joint unit 130 in the armdevice 10. Then, the control device 20 performs various operationsregarding the whole body coordination control using the generalizedinverse dynamics for controlling the driving of the arm unit 120 on thebasis of the state of the joint unit 130, and the motion purpose and theconstraint condition, and calculates the torque command value τ as theoperation result. Moreover, the arm device 10 controls the driving ofthe arm unit 120 on the basis of the torque command value τ. Asdescribed above, in the present embodiment, the driving of the arm unit120 is controlled by the whole body coordination control using thegeneralized inverse dynamics. Therefore, the drive control of the armunit 120 by force control is realized, and an arm device with higheroperability for the user is realized. Furthermore, in the presentembodiment, control to realize various motion purposes for furtherimproving the convenience of the user, such as the pivot operation andthe power assist operation, is possible in the whole body coordinationcontrol. Moreover, in the present embodiment, various driving means arerealized, such as manually moving the arm unit 120, and moving the armunit 120 by the operation input from a pedal. Therefore, furtherimprovement of the convenience for the user is realized.

Furthermore, in the present embodiment, the ideal joint control isapplied together with the whole body coordination control to the drivecontrol of the arm unit 120. In the ideal joint control, the disturbancecomponents such as friction and inertia inside the joint unit 130 areestimated, and the feedforward control using the estimated disturbancecomponents is performed. Therefore, even in a case where there is adisturbance component such as friction, an ideal response can berealized for the driving of the joint unit 130. Therefore, in the drivecontrol of the arm unit 120, highly accurate response and highpositioning accuracy and stability with less influence of vibration andthe like are realized.

Moreover, in the present embodiment, each of the plurality of jointunits 130 configuring the arm unit 120 has a configuration adapted tothe ideal joint control, and the rotation angle, generated torque andviscous drag coefficient in each joint unit 130 can be controlled withthe current value. As described above, the driving of each joint unit130 is controlled with the current value, and the driving of each jointunit 130 is controlled while grasping the state of the entire arm unit120 by the whole body coordination control. Therefore, counterbalance isunnecessary and downsizing of the arm device 10 is realized.

3. Configuration Example of Microsurgical System

The technology according to the present disclosure can be applied tovarious products. For example, the technology according to the presentdisclosure may be applied to a microsurgical system used for so-calledmicrosurgery, which is performed while performing close-up observationof a patient's minute site.

FIG. 6 is a view illustrating an example of a schematic configuration ofa microsurgical system 5300 to which the technology according to thepresent disclosure is applicable. Referring to FIG. 6, the microsurgicalsystem 5300 includes a microscope device 5301, a control device 5317,and a display device 5319. Note that, in the following description ofthe microsurgical system 5300, “user” means any medical staff who usesthe microsurgical system 5300, such as an operator or an assistant.

The microscope device 5301 includes a microscope unit 5303 formagnifying and observing an observation target (an operation site of apatient), an arm unit 5309 supporting the microscope unit 5303 at adistal end, and a base unit 5315 supporting a proximal end of the armunit 5309.

The microscope unit 5303 includes a substantially cylindrical tubularportion 5305, an imaging unit (not illustrated) provided inside thetubular portion 5305, and an operation portion 5307 provided in apartial region of an outer periphery of the tubular portion 5305. Themicroscope unit 5303 is an electronic imaging microscope unit (so-calledvideo microscope unit) that electronically captures a captured image bythe imaging unit.

A cover glass for protecting the imaging unit inside is provided on anopening surface in a lower end of the tubular portion 5305. Light fromthe observation target (hereinafter, also referred to as observationlight) passes through the cover glass and enters the imaging unit insidethe tubular portion 5305. Note that a light source including, forexample, a light emitting diode (LED) and the like may be providedinside the tubular portion 5305, and the observation target may beirradiated with light from the light source via the cover glass at thetime of imaging.

The imaging unit includes an optical system that condenses theobservation light and an imaging element that receives the observationlight condensed by the optical system. The optical system is configuredby a combination of a plurality of lenses including a zoom lens and afocus lens, and optical characteristics of the optical system areadjusted so as to focus the observation light on a light receivingsurface of the imaging element. The imaging element receives andphotoelectrically converts the observation light to generate a signalcorresponding to the observation light, in other words, an image signalcorresponding to an observed image. As the imaging element, for example,an imaging element capable of capturing a color image including a Bayerarray is used. The imaging element may be various known imaging elementssuch as a complementary metal oxide semiconductor (CMOS) image sensor ora charge coupled device (CCD) image sensor. The image signal generatedby the imaging element is transmitted to the control device 5317 as rawdata. Here, the transmission of the image signal may be suitablyperformed by optical communication. This is because, in a surgical site,the operator performs surgery while observing a state of an affectedpart with the captured image, and thus display of a moving image of anoperation site in as real time as possible is demanded for more safe andreliable surgery. When the image signal is transmitted by opticalcommunication, the captured image becomes able to be displayed with lowlatency.

Note that the imaging unit may have a drive mechanism that moves thezoom lens and the focus lens of the optical system along an opticalaxis. By appropriately moving the zoom lens and the focus lens by thedrive mechanism, magnification and a focal length at the time of imagingof the captured image can be adjusted. Furthermore, the imaging unit maybe equipped with various functions that can be generally provided in anelectronic imaging microscope unit, such as an auto exposure (AE)function and an auto focus (AF) function.

Furthermore, the imaging unit may be configured as a so-calledsingle-plate imaging unit including one imaging device, or may beconfigured as a so-called multi-plate imaging unit including a pluralityof imaging elements. In a case where the imaging unit is configured as amulti-plate imaging unit, for example, a color image may be obtained bygenerating image signals respectively corresponding to RGB by theimaging elements and synthesizing the image signals. Alternatively, theimaging unit may include a pair of imaging elements for respectivelyobtaining image signals for right eye and for left eye corresponding tothree-dimensional (3D) display. With the 3D display, the operator canmore accurately grasp the depth of biological tissue in the operationsite. Note that, in a case where the imaging unit is configured as amulti-plate imaging unit, a plurality of systems of the optical systemcan be provided corresponding to the imaging elements.

The operation portion 5307 includes, for example, a cross lever, aswitch, or the like, and is input means that receives a user's operationinput. For example, the user can input an instruction to change themagnification of the observed image and the focal length to theobservation target via the operation portion 5307. By appropriatelymoving the zoom lens and the focus lens by the drive mechanism of theimaging unit according to the instruction, the magnification and thefocal length can be adjusted. Furthermore, for example, the user caninput an instruction to switch an operation mode (all free mode andfixed mode described below) of the arm unit 5309 via the operationportion 5307. Note that, in a case where the user tries to move themicroscope unit 5303, a mode in which the user moves the microscope unit5303 in a state of holding the tubular portion 5305 is assumed.Therefore, it is favorable to provide the operation portion 5307 at aposition where the user can easily operate the operation portion 5307with a finger in a state of holding the tubular portion 5305 so that theuser can operate the operation portion 5307 while moving the tubularportion 5305.

The arm unit 5309 is configured such that a plurality of links (firstlink 5313 a to sixth link 5313 f) is rotatably connected with oneanother by a plurality of joint units (first joint unit 5311 a to sixthjoint unit 5311 f).

The first joint unit 5311 a has a substantially cylindrical shape, and adistal end (lower end) of the first joint unit 5311 a rotatably supportsan upper end of the tubular portion 5305 of the microscope unit 5303around a rotation axis (first axis O₁) parallel to a central axis of thetubular portion 5305. Here, the first joint unit 5311 a can beconfigured such that the first axis O₁ coincides with an optical axis ofthe imaging unit of the microscope unit 5303. With the configuration, afield of view can be changed to rotate a captured image by rotating themicroscope unit 5303 around the first axis O₁.

The first link 5313 a fixedly supports the first joint unit 5311 a atthe distal end. Specifically, the first link 5313 a is a rod-like memberhaving a substantially L-shape, and is connected to the first joint unit5311 a such that an end portion of one side on a distal end side comesin contact with an upper end portion of an outer periphery of the firstjoint unit 5311 a while the one side on the distal end side extends in adirection orthogonal to the first axis O₁. The second joint unit 5311 bis connected to an end portion of the other side on a proximal end sideof the substantially L-shape of the first link 5313 a.

The second joint unit 5311 b has a substantially cylindrical shape, anda distal end of the second joint unit 5311 b rotatably supports theproximal end of the first link 5313 a around a rotation axis (secondaxis O₂) orthogonal to the first axis O₁. A distal end of the secondlink 5313 b is fixedly connected to a proximal end of the second jointunit 5311 b.

The second link 5313 b is a rod-like member having a substantiallyL-shape, and an end portion of one side on a distal end side is fixedlyconnected to the proximal end of the second joint unit 5311 b while theone side extends in a direction orthogonal to the second axis O₂. Thethird joint unit 5311 c is connected to the other side on a proximal endside of the substantially L-shape of the second link 5313 b.

The third joint unit 5311 c has a substantially cylindrical shape, and adistal end of the third joint unit 5311 c rotatably supports theproximal end of the second link 5313 b around a rotation axis (thirdaxis O₃) orthogonal to the first axis O₁ and the second axis O₂. Adistal end of the third link 5313 c is fixedly connected to a proximalend of the third joint unit 5311 c. The microscope unit 5303 can bemoved to change the position of the microscope unit 5303 in a horizontalplane by rotating the configuration on the distal end side including themicroscope unit 5303 around the second axis O₂ and the third axis O₃.That is, by controlling the rotation around the second axis O₂ and thethird axis O₃, the field of view of the captured image can be moved inthe plane.

The third link 5313 c is configured such that its distal end side has asubstantially cylindrical shape, and the proximal end of the third jointunit 5311 c is fixedly connected to the distal end of the cylindricalshape such that both the third joint unit 5311 c and the cylindricalshape have substantially the same central axis. A proximal end side ofthe third link 5313 c has a prismatic shape, and the fourth joint unit5311 d is connected to an end portion on the proximal end side.

The fourth joint unit 5311 d has a substantially cylindrical shape, anda distal end of the fourth joint unit 5311 d rotatably supports theproximal end of the third link 5313 c around a rotation axis (fourthaxis O₄) orthogonal to the third axis O₃ A distal end of the fourth link5313 d is fixedly connected to a proximal end of the fourth joint unit5311 d.

The fourth link 5313 d is a substantially linearly extending rod-likemember, and is fixedly connected to the fourth joint unit 5311 d whileextending to be orthogonal to the fourth axis O₄ such that an endportion on a distal end of the fourth link 5313 d comes in contact witha side surface of the substantially cylindrical shape of the fourthjoint unit 5311 d. The fifth joint unit 5311 e is connected to aproximal end of the fourth link 5313 d.

The fifth joint unit 5311 e has a substantially cylindrical shape, and adistal end of the fifth joint unit 5311 e rotatably supports theproximal end of the fourth link 5313 d around a rotation axis (fifthaxis O₅) parallel to the fourth axis O₄. A distal end of the fifth link5313 e is fixedly connected to a proximal end of the fifth joint unit5311 e. The fourth axis O₄ and the fifth axis O₅ are rotation axesenabling the microscope unit 5303 to move in the up-down direction. Theheight of the microscope unit 5303, in other words, the distance betweenthe microscope unit 5303 and the observation target can be adjusted byrotating the configuration on the distal end side including themicroscope unit 5303 around the fourth axis O₄ and the fifth axis O₅.

The fifth link 5313 e is configured by a combination of a first memberhaving a substantially L-shape and having one side extending in avertical direction and the other side extending in a horizontaldirection, and a rod-like second member extending vertically downwardfrom a portion of the first member extending in the horizontaldirection. The proximal end of the fifth joint unit 5311 e is fixedlyconnected to a vicinity of an upper end of the portion of the firstmember of the fifth link 5313 e, the portion extending in the verticaldirection. The sixth joint unit 5311 f is connected to a proximal end(lower end) of the second member of the fifth link 5313 e.

The sixth joint unit 5311 f has a substantially cylindrical shape, and adistal end of the sixth joint unit 5311 f rotatably supports theproximal end of the fifth link 5313 e around a rotation axis (sixth axisO₆) parallel to the vertical direction. A distal end of the sixth link5313 f is fixedly connected to a proximal end of the sixth joint unit5311 f.

The sixth link 5313 f is a rod-like member extending in the verticaldirection, and the proximal end of the sixth link 5313 f is fixedlyconnected to an upper surface of the base unit 5315.

Rotatable ranges of the first joint unit 5311 a to sixth joint unit 5311f are appropriately set such that the microscope unit 5303 can performdesired movement. Thereby, in the arm unit 5309 having theabove-described configuration, the movement of a total of six degrees offreedom, i.e., three translational degrees of freedom and threerotational degrees of freedom, can be realized with respect to themovement of the microscope unit 5303. By configuring the arm unit 5309to realize six degrees of freedom with respect to the movement of themicroscope unit 5303 in this way, the position and posture of themicroscope unit 5303 can be freely controlled within the movable rangeof the arm unit 5309. Therefore, the operation site can be observed fromany angle, and the surgery can be more smoothly executed.

Note that the configuration of the illustrated arm unit 5309 is merelyan example, and the number and shapes (lengths) of the links configuringthe arm unit 5309, and the number, arrangement positions, directions ofthe rotation axes, and the like of the joint units may be appropriatelydesigned to realize desired degrees of freedom. For example, asdescribed above, to freely move the microscope unit 5303, the arm unit5309 is favorably configured to have six degrees of freedom. However,the arm unit 5309 may be configured to have larger degrees of freedom(in other words, redundant degrees of freedom). In a case where theredundant degrees of freedom exist, in the arm unit 5309, the posture ofthe arm unit 5309 can be changed in a case where the position andposture of the microscope unit 5303 are fixed. Therefore, moreconvenient control for the operator can be realized, such as controllingthe posture of the arm unit 5309 such that the arm unit 5309 does notinterfere with the field of vision of the operator looking at thedisplay device 5319, for example.

Here, each of the first joint unit 5311 a to sixth joint unit 5311 f canbe provided with an actuator mounted with a drive mechanism such as amotor, an encoder for detecting the rotation angle in each joint unit,and the like. Then, the posture of the arm unit 5309, in other words,the position and posture of the microscope unit 5303 can be controlledas driving of the actuators provided in the first joint unit 5311 a tosixth joint unit 5311 f are appropriately controlled by the controldevice 5317. Specifically, the control device 5317 can grasp the currentposture of the arm unit 5309, and the current position and posture ofthe microscope unit 5303, on the basis of information of the rotationangles of the joint units detected by the encoders. The control device5317 calculates control values (for example, rotation angles orgenerated torques, and the like) for the joint units that realize themovement of the microscope unit 5303 according to the operation inputfrom the user, using the grasped information grasped, and drives thedrive mechanisms of the joint units according to the control values.Note that, at this time, the method of controlling the arm unit 5309 bythe control device 5317 is not limited, and various known controlmethods such as force control or position control may be applied.

For example, the driving of the arm unit 5309 may be appropriatelycontrolled by the control device 5317 according to the operation input,and the position and posture of the microscope unit 5303 may becontrolled by the operator appropriately performing an operation inputvia an input device (not illustrated). With the control, the microscopeunit 5303 can be moved from an arbitrary position to an arbitraryposition, and then can be fixedly supported at the position after themovement. Note that, as the input device, an input device operable bythe operator, such as a foot switch, for example, even if the operatorholds a treatment tool, is favorably applied. Furthermore, the operationinput may be performed in a non-contact manner on the basis of gesturedetection or gaze detection using a camera provided on a wearable deviceor in the operating room. As a result, even a user belonging to a cleanarea can operate devices belonging to a filthy area with a higher degreeof freedom. Alternatively, the arm unit 5309 may be operated by aso-called master-slave system. In this case, the arm unit 5309 can beremotely operated by the user via the input device installed at a placedistant from an operating room.

Furthermore, in a case where the force control is applied, so-calledpower assist control in which an external force is received from theuser, and the actuators of the first joint unit 5311 a to sixth jointunit 5311 f are driven such that the arm unit 5309 smoothly moves alongthe external force may be performed. With the control, the user can movethe microscope unit 5303 with a relatively light force when directlymoving while holding the microscope unit 5303 to another position.Accordingly, the user can more intuitively move the microscope unit 5303with a simpler operation, and the user's convenience can be improved.

Furthermore, the driving of the arm unit 5309 may be controlled toperform a pivot operation. Here, the pivot operation is an operation tomove the microscope unit 5303 such that the optical axis of themicroscope unit 5303 always faces a predetermined point in the space(hereinafter, referred to as a pivot point). According to the pivotoperation, the same observation position can be observed from variousdirections, and more detailed observation of the affected part becomespossible. Note that, in a case where the microscope unit 5303 isconfigured to be unable to adjust its focal length, it is favorably toperform the pivot operation in a case where the distance between themicroscope unit 5303 and the pivot point is fixed. In this case, it issufficient that the distance between the microscope unit 5303 and thepivot point is adjusted to a fixed focal length of the microscope unit5303. With the adjustment, the microscope unit 5303 moves on ahemispherical surface (schematically illustrated in FIG. 6) having aradius corresponding to the focal length centered on the pivot point,and a clear captured image can be obtained even if the observationdirection is changed. Meanwhile, in a case where the microscope unit5303 is configured to be able to adjust its focal length, the pivotoperation may be performed in a case where the distance between themicroscope unit 5303 and the pivot point is variable. In this case, forexample, the control device 5317 calculates the distance between themicroscope unit 5303 and the pivot point on the basis of the informationof the rotation angles of the joint units detected by the encoders, andmay automatically adjusted the focal length of the microscope unit 5303on the basis of the calculation result. Alternatively, in a case wherethe microscope unit 5303 is provided with an AF function, the focallength may be automatically adjusted by the AF function each time thedistance between the microscope unit 5303 and the pivot point changesdue to the pivot operation.

Furthermore, the first joint unit 5311 a to sixth joint unit 5311 f maybe provided with brakes that restrict the rotation thereof. Operationsof the brakes may be controlled by the control device 5317. For example,in a case where it is desired to fix the position and posture of themicroscope unit 5303, the control device 5317 actuates the brakes of therespective joint units. Accordingly, the posture of the arm unit 5309,in other words, the position and posture of the microscope unit 5303 canbe fixed even if the actuators are not driven. Therefore, the powerconsumption can be decreased. In a case where it is desired to move theposition and posture of the microscope unit 5303, it is sufficient thatthe control device 5317 releases the brakes of the joint units anddrives the actuators according to a predetermined control method.

Such an operation of the brakes can be performed in response to anoperation input by the user via the above-described operation portion5307. In a case where the user wants to move the position and posture ofthe microscope unit 5303, the user operates the operation portion 5307to release the brakes of the respective joint units. As a result, theoperation mode of the arm unit 5309 shifts to a mode (all free mode) inwhich rotation in the joint units can be freely performed. Furthermore,in a case where the user wants to fix the position and posture of themicroscope unit 5303, the user operates the operation portion 5307 toactuate the brakes of the respective joint units. As a result, theoperation mode of the arm unit 5309 shifts to a mode (fixed mode) inwhich the rotation in the joint units is restricted.

The control device 5317 controls operations of the microscope device5301 and the display device 5319 to centrally control the operation ofthe microsurgical system 5300. For example, the control device 5317operates the actuators of the first joint unit 5311 a to sixth jointunit 5311 f according to a predetermined control method to control thedriving of the arm unit 5309. Furthermore, for example, the controldevice 5317 controls the operations of the brakes of the first jointunit 5311 a to sixth joint unit 5311 f to change the operation mode ofthe arm unit 5309. Furthermore, for example, the control device 5317applies various types of signal processing to the image signal acquiredby the imaging unit of the microscope unit 5303 of the microscope device5301 to generate image data for display and to cause the display device5319 to display the image data. In the image processing, various typesof known signal processing may be performed, such as developmentprocessing (demosaicing processing), high image quality processing (bandenhancement processing, super resolution processing, noise reduction(NR) processing, and/or camera shake correction processing, forexample), and/or enlargement processing (in other words, electronic zoomprocessing), for example.

Note that the communication between the control device 5317 and themicroscope unit 5303, and the communication between the control device5317 and the first joint unit 5311 a to sixth joint unit 5311 f may bewired communication or wireless communication. In the case of wiredcommunication, communication by electrical signals may be performed oroptical communication may be performed. In this case, a transmissioncable used for the wired communication can be configured as anelectrical signal cable, an optical fiber, or a composite cable of theaforementioned cables according to the communication system. Meanwhile,in the case of wireless communication, since it is not necessary to laythe transmission cable in the operating room, the situation in whichmovement of the medical staff in the operating room is hindered by thetransmission cable can be resolved.

The control device 5317 can be a processor such as a central processingunit (CPU) or a graphics processing unit (GPU), a microcomputer or acontrol board in which a processor and a memory element such as a memoryare mixed, or the like.

The processor of the control device 5317 is operated according to apredetermined program, whereby the above-described various functions canbe realized. Note that, in the illustrated example, the control device5317 is provided as a separate device from the microscope device 5301.However, the control device 5317 may be installed inside the base unit5315 of the microscope device 5301 and integrally configured with themicroscope device 5301. Alternatively, the control device 5317 may beconfigured by a plurality of devices. For example, microcomputers,control boards, or the like are respectively disposed in the microscopeunit 5303 and the first joint unit 5311 a to sixth joint unit 5311 f ofthe arm unit 5309, and are communicatively connected to one another,whereby similar functions to the control device 5317 may be realized.

The display device 5319 is provided in the operating room, and displaysan image corresponding to the image data generated by the control device5317 under the control of the control device 5317. That is, an image ofthe operation site captured by the microscope unit 5303 is displayed onthe display device 5319. Note that the display device 5319 may displayvarious types of information regarding surgery, such as physicalinformation of the patient and information regarding an operationtechnique of the surgery, for example, in place of or in addition to theimage of the operation site. In this case, the display of the displaydevice 5319 may be appropriately switched by the operation by the user.Alternatively, a plurality of the display devices 5319 may be provided,and the image of the operation site and the various types of informationregarding surgery may be displayed on each of the plurality of displaydevices 5319.

Note that, as the display device 5319, various known display devicessuch as a liquid crystal display device or an electro luminescence (EL)display device may be applied.

4. Estimation of Force Acting from Disturbance According to PresentEmbodiment

In the present embodiment, formulation of the estimation of the forceacting from the disturbance is performed by considering an actualsurgical scene (including the relationship with surgical tools otherthan the arm or the like) and environmental conditions. Thereby,estimation of forces of various types of disturbance acting on the armin the surgical environment can be realized. Thereby, application ofsafety stop by contact detection or switching of the control state ofthe arm by the operation force detection or the like to the userinterface, and implementation of an application such as presentation offorce sense to the outside become possible.

Such technology is different from the conventional technology by “(1)physical and structural characteristics of the robot”. Furthermore, suchtechnology is different from the conventional technology by “(2) thecharacteristics that the external force due to main disturbance (tensiondue to the light source and the camera cable) is limited, and pointswhere other types of disturbance act can be assumed”.

First, the above-described “(1) physical and structural characteristicsof the robot” will be described. First, the technology according to thepresent embodiment is different from the conventional technology using amedical arm in that installation of a force sensor is not necessary.Furthermore, second, the technology according to the present embodimentis different from the conventional technology using a general-purposearm in that a decrease in force estimation accuracy due to smallmanipulability can be avoided in a body structure assuming an operationsite and an operation method.

Subsequently, “(2) the characteristics that the external force due tomain disturbance (tension due to the light source and the camera cable)is limited, and points where other types of disturbance act can beassumed” will be described. According to the technology of the presentembodiment, the disturbance due to the cable can be estimated, and thecontrol and force assist for compensating the disturbance can beperformed. In addition, the disturbance other than due to the cable,which acts on any observation point on the body can be estimated.

Specifically, in the arm control system according to the presentembodiment, the joint state acquisition unit 241 acquires the state ofthe joint unit 130 of the arm unit 120. Then, the disturbance estimationunit 251 estimates the external force due to the disturbance on thebasis of the condition that the external force due to the predetermineddisturbance is limited to one predetermined direction or a plurality ofpredetermined directions, and the state of the joint unit 130. In otherwords, the disturbance estimation unit 251 estimates the external forcedue to the disturbance on the basis of the state of the joint unit 130after limiting the direction of the external force to be detected to theone predetermined direction or the plurality of predetermineddirections.

At this time, the external force estimation unit 251 estimates theexternal force acting on a predetermined observation point. Note thatthe one predetermined direction or a plurality of predetermineddirections in which the external force is limited can include a rotationdirection (moment direction) in addition to a translational direction.Furthermore, the arm control system can function as a medical supportarm system in a case of being applied to the medical field. Furthermore,hereinafter, a hard endoscope will be mainly described as an example ofan endoscope, but a soft endoscope may be used instead of the hardendoscope.

More specifically, according to the technology of the presentembodiment, there are effects such as “(2-1) the observation point isplaced on the camera head, and the external force can be perceived ashuman operation force”, “(2-2) the observation point is placed at thedistal end of the hard endoscope, and a contact collision of the distalend of the hard endoscope can be detected”, “(2-3) the observation pointis placed at the trocar point, and the force acting from the trocar canbe perceived”, and “(2-4) in above (2-1) to (2-3), “(A) by placingrestrictions on the perceived disturbance”, “(B) or by giving redundancyto arm degrees of freedom”, “(C) or by installing a force sensor at aspecific part of the distal end”, the operation force and the contactand collision can be detected in a complex manner”.

FIG. 9 is a view for describing an example of a force acting from atrocar point. Referring to FIG. 9, a hard endoscope unit 425 isillustrated. Furthermore, a trocar point 71 at which the hard endoscopeunit 425 is inserted into the body cavity of the patient 70 is shown. Inthe example illustrated in FIG. 9, the external force acting on the hardendoscope unit 425 is constrained by the trocar point 71. Morespecifically, illustrated in FIG. 9, the external force acting on thehard endoscope unit 425 is limited to a pitch direction, a rolldirection, and a zoom direction.

Hereinafter, specific examples will be described. The external torqueτ_(n) observed by the VA installed in each joint unit is expressed asthe following (13).

[Math. 12]

τ_(i) :i∈[1,n]  (13)

Note that, in FIG. 3, the external torques observed by the VA installedin the joint units are represented as τ₁, τ₂, τ₃, . . . , and τ_(n).Furthermore, the external forces acting on respective observation pointsare represented as follows.

f_(cable): Tension (fx, fy, fz) by the cable

f_(op): Tension (fx, fy, fz) by the hand holding the camera head

f_(trocar): Force (fx, fy) acting from the trocar

f_(tip): Force (fx, fy, fz) acting on the distal end of the hardendoscope

Note that, in FIG. 3, the external forces acting on the respectiveobservation points are represented as f_(cable), f_(op), f_(trocar), . .. , and f_(tip). Furthermore, a basic expression representing therelationship between the external force and the external torque isexpressed as in (14) below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 13} \right\rbrack & \; \\{\tau^{T} = {\begin{bmatrix}J_{tip} \\J_{trocar} \\J_{op} \\J_{cable}\end{bmatrix}^{T}\begin{bmatrix}{J_{tip}J_{trocar}} & {J_{op}J_{cable}}\end{bmatrix}}^{T}} & (14)\end{matrix}$

Note that, to measure all the forces, torque values for eleven axes arerequired. Furthermore, in a case of using a moment as an operationforce, torque values for a maximum of fourteen axes are required.Furthermore, full force sense can be detected with a configuration ofeight-axis redundant degrees of freedom arm six-axis force sensor.

Here, the reason why the tension by the cable is limited will bedescribed. FIG. 7 is a view illustrating an appearance of the hardendoscope unit. As illustrated in FIG. 7, the hard endoscope unit 425includes a hard endoscope main body 426, a cable 424, and a connectionportion 432. The hard endoscope main body 426 includes a camera head CH.Furthermore, the hard endoscope main body 426 includes a holding portion431 held by the arm. The connection portion 432 is a connection portionbetween the hard endoscope main body 426 and the cable 424.

FIG. 8 is an enlarged view of the connection portion 432. The connectionportion 432 includes a cable connection component 433 for connecting thehard endoscope main body 426 and the cable 424. The cable connectioncomponent 433 is a rigid body. Here, the value of the external moment atthe connection portion 432 is considered to be extremely smaller thanother types of disturbance, for the reasons described below.

Reason 1: The connection portion between the cable connection component433 and the hard endoscope main body 426 is designed to be free (to makefriction small) with respect to rotation in a direction M1.

Reason 2: The moment arm is extremely short (for example, at most about5 [mm] equal to the radius of the cable 424) in the connection portionbetween the cable connection component 433 and the cable 424.

In light of the foregoing, the disturbance acting on the hard endoscopeunit 425 from the cable 424 can be regarded as a force in a triaxialhorizontal direction. Note that FIG. 8 illustrates directions in whichthe moment generated by the disturbance is considered to be small due tostructural reasons (the direction M1, a direction M2, and a directionM3).

From the formulation of disturbance estimation using the selectionmatrix described in (14), the following (15) is established.

$\begin{matrix}{\mspace{79mu} \left\lbrack {{Math}.\mspace{14mu} 14} \right\rbrack} & \; \\{{\begin{bmatrix}{f_{tip}f_{trocar}} & {f_{op}f_{cable}}\end{bmatrix}^{T}\begin{bmatrix}k_{1} & \ldots & 0 \\\vdots & \ddots & \vdots \\0 & \ldots & k_{x}\end{bmatrix}} = {\left( {\begin{bmatrix}J_{tip} \\J_{trocar} \\J_{op} \\J_{cable}\end{bmatrix}^{T}\begin{bmatrix}k_{1} & \ldots & 0 \\\vdots & \ddots & \vdots \\0 & \ldots & k_{x}\end{bmatrix}} \right)^{\#}\tau^{T}}} & (15)\end{matrix}$

Here, in a case where the arm is an endoscope arm that supports anendoscope device, the disturbance estimation unit 251 can estimate thedisturbance by the following (16) at the time of an assist operation.

$\begin{matrix}{\; \left\lbrack {{Math}.\mspace{14mu} 15} \right\rbrack} & \; \\{{\left\lbrack {f_{op}f_{cable}} \right\rbrack^{T}\begin{bmatrix}k_{1} & \ldots & 0 \\\vdots & \ddots & \vdots \\0 & \ldots & k_{x}\end{bmatrix}} = \begin{bmatrix}\lbrack 0\rbrack & \; & \; & 0 \\\; & \lbrack 0\rbrack & \; & \; \\\; & \; & \lbrack 1\rbrack & \; \\0 & \; & \; & \lbrack 1\rbrack\end{bmatrix}} & (16)\end{matrix}$

Furthermore, in a case where the arm is an endoscope arm that supportsan endoscope device, the disturbance estimation unit 251 can estimatethe disturbance by the following (17) at the time of an assistoperation.

$\begin{matrix}{\; \left\lbrack {{Math}.\mspace{14mu} 16} \right\rbrack} & \; \\{{\begin{bmatrix}f_{tip} & f_{cable}\end{bmatrix}^{T}\begin{bmatrix}k_{1} & \ldots & 0 \\\vdots & \ddots & \vdots \\0 & \ldots & k_{x}\end{bmatrix}} = \begin{bmatrix}\lbrack 1\rbrack & \; & \; & 0 \\\; & \lbrack 0\rbrack & \; & \; \\\; & \; & \lbrack 0\rbrack & \; \\0 & \; & \; & \lbrack 1\rbrack\end{bmatrix}} & (17)\end{matrix}$

Furthermore, in a case where the arm is a camera arm supporting a camera(for example, a microscope unit), the disturbance estimation unit 251can estimate a force received from a monitor that displays navigationinformation and a force operated by a person by the following (18) atthe time of an assist operation.

$\begin{matrix}{\; \left\lbrack {{Math}.\mspace{14mu} 17} \right\rbrack} & \; \\{{\begin{bmatrix}f_{tip} & f_{op}\end{bmatrix}^{T}\begin{bmatrix}k_{1} & \ldots & 0 \\\vdots & \ddots & \vdots \\0 & \ldots & k_{x}\end{bmatrix}} = \begin{bmatrix}\lbrack 1\rbrack & \; & \; & 0 \\\; & \lbrack 0\rbrack & \; & \; \\\; & \; & \lbrack 1\rbrack & \; \\0 & \; & \; & \lbrack 0\rbrack\end{bmatrix}} & (18)\end{matrix}$

Furthermore, in a case where the arm is a camera arm supporting a camera(for example, a microscope unit), the disturbance estimation unit 251can estimate a force received from a monitor that displays navigationinformation by the following (19) at the time of a remote operation.

$\begin{matrix}{\; \left\lbrack {{Math}.\mspace{14mu} 18} \right\rbrack} & \; \\{{\left\lbrack f_{tip} \right\rbrack^{T}\begin{bmatrix}k_{1} & \ldots & 0 \\\vdots & \ddots & \vdots \\0 & \ldots & k_{x}\end{bmatrix}} = \begin{bmatrix}\lbrack 1\rbrack & \; & \; & 0 \\\; & \lbrack 0\rbrack & \; & \; \\\; & \; & \lbrack 0\rbrack & \; \\0 & \; & \; & \lbrack 0\rbrack\end{bmatrix}} & (19)\end{matrix}$

Note that a surgical navigation system may be connected to the cameraarm as an external device. In a case of introducing a navigation system,a monitor for navigation and the like are installed (connected) to thearm. With the installation, the arm's own weight (including physicaldata) deviates from a design value and a negative effect on forcecontrol is predicted. By estimating the force received from the monitoras an external force according to this idea, change from the designedvalue of the arm's own weight can be compensated. Note that the surgicalnavigation system may also be included in the medical support arm systemaccording to the present embodiment.

Furthermore, in a case where the arm is an arm supporting a retractor ora forceps, the disturbance estimation unit 251 can estimate a forcereceived from the monitor that displays navigation information and aforce of the person (operator) who operates the retractor or the forcepsby the following (20) at the time of an assist operation, using f_(op)as the force of the operator who operates the retractor, the forceps, orthe like.

$\begin{matrix}{\; \left\lbrack {{Math}.\mspace{14mu} 19} \right\rbrack} & \; \\{{\begin{bmatrix}f_{tip} & f_{op}\end{bmatrix}^{T}\begin{bmatrix}k_{1} & \ldots & 0 \\\vdots & \ddots & \vdots \\0 & \ldots & k_{x}\end{bmatrix}} = \begin{bmatrix}\lbrack 1\rbrack & \; & \; & 0 \\\; & \lbrack 0\rbrack & \; & \; \\\; & \; & \lbrack 1\rbrack & \; \\0 & \; & \; & \lbrack 0\rbrack\end{bmatrix}} & (20)\end{matrix}$

Furthermore, in a case where the arm is an arm supporting a retractor ora forceps, the disturbance estimation unit 251 can estimate a forcereceived from a monitor that displays navigation information by thefollowing (21) at the time of a remote operation.

$\begin{matrix}{\; \left\lbrack {{Math}.\mspace{14mu} 20} \right\rbrack} & \; \\{{\left\lbrack f_{tip} \right\rbrack^{T}\begin{bmatrix}k_{1} & \ldots & 0 \\\vdots & \ddots & \vdots \\0 & \ldots & k_{x}\end{bmatrix}} = \begin{bmatrix}\lbrack 1\rbrack & \; & \; & 0 \\\; & \lbrack 0\rbrack & \; & \; \\\; & \; & \lbrack 0\rbrack & \; \\0 & \; & \; & \lbrack 0\rbrack\end{bmatrix}} & (21)\end{matrix}$

Note that, when it is considered that the force acting on the distal endis narrowed to a translational force in view of the fact that thedistance (moment arm) from an acting point to the observation point islong, the system cannot be an underestimation system. Here, theunderestimation system refers to a system in which the number of unknownvariables to be estimated exceeds the number of measurable variables,and the values of the unknown variables cannot be uniquely determined(estimated).

The specific disturbance estimation has been described above.

5. Joint Control According to External Force According to PresentEmbodiment

When the external force is estimated by the disturbance estimation unit251, the command value calculation unit 252 controls the joint unit 130according to the estimated external force. The command value calculationunit 252 can function as a joint control unit. For example, in a casewhere the observation point is placed on the distal end of the hardendoscope, the disturbance estimation unit 251 estimates the externalforce acting on the distal end of the hard endoscope, and the commandvalue calculation unit 252 controls the joint unit 130 according to theexternal force. Such an example will be described with reference toFIGS. 10 and 11.

FIGS. 10 and 11 are views for describing an example of joint control ina case where an observation point is placed at a distal end of the hardendoscope. Referring to FIG. 10, a hard endoscope 425-1 is moved to ahard endoscope 425-2 by the arm unit 120. In the example illustrated inFIG. 10, an external force F1 acts on the distal end of the hardendoscope. Here, assuming a case in which the distal end of the hardendoscope interferes with a tissue 72 of a patient 70, as illustrated inFIG. 11, an external force F2 and an external force F3 acting on thedistal end of the hard endoscope from the tissue 72 are estimated by thedisturbance estimation unit 251.

At this time, the command value calculation unit 252 controls the jointunit 130 such that the arm unit 120 moves in a direction according to adirection of the external force F2 or the external force F3, or the armunit 120 stops. With the control, even in a case where the hardendoscope is mistakenly operated and the distal end of the hardendoscope is brought in contact with the tissue 72 to harm the patient,the external force acting on the distal end of the hard endoscope isrecognized, and the arm unit 120 is stopped or avoided to a safedirection, for example, whereby the safety at the time of surgery can beincreased. The direction according to the direction of the externalforce F2 or the external force F3 may be the same as the external forceF2, or may be the direction according to the direction of the externalforce F3.

Meanwhile, for example, in a case where the observation point is placedon the camera head, the disturbance estimation unit 251 estimates theexternal force acting on the camera head, and the command valuecalculation unit 252 controls the joint unit 130 according to theexternal force. Specifically, in a case where it is estimated that anexternal force acts on the camera head, the command value calculationunit 252 controls the joint unit 130 such that the arm unit 120 moves ina direction according to the direction of the external force. Thedirection according to the direction of the external force may be thesame direction as the external force. In doing so, the arm unit 120 ismoved in the direction intended by the operator.

It is difficult for the operator (surgeon) to receive a sense of forceor touch in a holding device of a master slave or remote controlendoscope. Therefore, the operator may recognize that the hard endoscopeis mistakenly giving a tissue a stress by an alert, so that theoperation can safely continue the surgery. Specifically, an outputcontrol unit 264 (FIG. 12) may control the alert to be output by anoutput device in a case where the external force exceeds a thresholdvalue. Alternatively, the output control unit 264 (FIG. 12) may performcontrol such that the magnitude of the external force or the directionof the external force is output by the output device.

Here, the output device may be the display device 30 that performsdisplay so as to be visually perceived by the operator, or may be anotification device 80 (FIG. 12). The notification device 80 includes atleast one of a sound output device (such as a buzzer) that outputs asound so that the operator or the surrounding medical staff audiblyperceives the sound by the operator, or a light output device (such as alamp) that outputs light. Furthermore, the alert may be stoppable by astop instruction input via the input unit 210.

The joint control according to the specific external force has beendescribed above.

6. Specific Configuration Example of Arm Control System

Next, a specific configuration example of the arm control system will bedescribed. FIG. 12 is a diagram illustrating a specific configurationexample of the arm control system. As illustrated in FIG. 12, the armcontrol system includes the arm unit 120, the control unit 230, theinput unit 210, the display device 30, and a notification device 80. Thefunctions of the input unit 210, the display device 30, and thenotification device 80 are as described above. Furthermore, the controlunit 230 includes a sensor information acquisition unit 261, an armstate acquisition unit 262, an external force estimation unit 263, aninput/output control unit 264, an operation determination unit 265, awhole body control unit 266, a joint control unit 267, and a drive unit268.

The sensor information acquisition unit 261 acquires the state of eachjoint (sensor information of the encoder and the torque sensor) of thearm unit 120, and outputs the state to the joint control unit 267 andthe arm state acquisition unit 241. The arm state acquisition unit 262can correspond to the arm state acquisition unit 241 illustrated in FIG.5. The external force estimation unit 263 can correspond to thedisturbance estimation unit 251 illustrated in FIG. 5. The input/outputcontrol unit 264 has a function to acquire input information from theinput unit 210, and a function to control outputs of output informationby the display device 30 and the notification device. The operationdetermination unit 265 can correspond to the arithmetic conditionsetting unit 242 illustrated in FIG. 5. The whole body control unit 266can correspond to the virtual force calculation unit 243 and the realforce calculation unit 244 illustrated in FIG. 5. The joint control unit267 can correspond to the command value calculation unit 252 illustratedin FIG. 5. The drive unit 268 can correspond to the drive control unit111 illustrated in FIG. 5.

According to the present embodiment, the direction of the external forcedetected by the disturbance estimation unit 251 is limited to onepredetermined direction or a plurality of predetermined directions.Thereby, the number of sensors can be reduced, and cost reduction can beperformed. Moreover, in a case where the arm control system is appliedto the medical field, the sensor is omitted in an area (area close tothe clean area close to the distal end of the arm) overlapping a workingarea of the operator (surgeon), and a simple structure can be realized.

For example, according to the present embodiment, the number of torquesensors provided in the arm unit 120 can be made smaller than the numberof joint units. In the example illustrated in FIG. 12, the arm unit 120is provided with six joint units, but the torque sensors are provided inonly three joint units out of the six joint units (torque sensors 614 ato 614 c).

Meanwhile, the encoders are provided in all the six joint units(encoders 613 a to 613 f). As such, the arm unit 120 may include anencoder having six degrees of freedom or an encoder having a degree offreedom larger than six degrees of freedom. Furthermore, the motors arealso provided in all the six joint units (motors 611 a to 611 f).

Furthermore, the arm unit 120 according to the present embodimentincludes at least three or more series of joint units, and torquesensors included in adjacent joint units of the three or more jointunits have independent degrees of freedom of each other. The torquesensors included in adjacent joint units having independent degrees offreedom of each other means that the rotation directions of the torquesensors of the adjacent joint units are not both the roll direction, thepitch direction, or the yaw direction. By configuring the torque sensorsof the adjacent joint units of the three or more joint units to haveindependent degrees of freedom of each other, the number of torquesensors provided in the arm unit 120 can be reduced to the numbersmaller than the number of joint units.

Specifically, in a case where the rotation directions of the torquesensors included the three or more joint units are “pitch, roll, yaw”,“roll, yaw, roll”, “pitch, roll, pitch”, or the like, the torque sensorsincluded in the adjacent joint units have independent degrees of freedomof each other. Meanwhile, in a case where the rotation directions of thetorque sensors included the three or more joint units are “pitch, pitch,yaw”, or the like, the torque sensors included in the adjacent jointunits do not have independent degrees of freedom of each other. In theexample illustrated in FIG. 3, in the case of “421 c, 421 d, 421 e”,“421 d, 421 e, 421 f”, or the like, the torque sensors included in theadjacent joint units have independent degrees of freedom of each other.

The specific configuration example of the arm control system has beendescribed above.

According to the present embodiment, provided is a medical support armsystem including a joint state acquisition unit configured to acquire astate of a joint unit of an arm unit, and an external force estimationunit configured to estimate an external force due to predetermineddisturbance on the basis of a condition that the external force due tothe predetermined disturbance is limited to one predetermined directionor a plurality of predetermined directions, and a state of the jointunit.

According to the configuration, in an endoscope holder arm, disturbancedue to tension of a camera and a light source cable can be compensated,and more robust control can be realized. Furthermore, according to sucha configuration, an operation force by a person or contact with theoutside is detected other than the disturbance due to tension of acamera and a light source cable, and the contact detection can be usedas an operation trigger.

Although the favorable embodiments of the present disclosure have beendescribed in detail with reference to the accompanying drawings, thetechnical scope of the present disclosure is not limited to suchexamples. It is obvious that persons having ordinary knowledge in thetechnical field of the present disclosure can conceive variousmodifications or alterations within the scope of the technical ideadescribed in the claims, and the modifications and alterations arenaturally understood to belong to the technical scope of the presentdisclosure.

Furthermore, the effects described in the present specification aremerely illustrative or exemplary and are not restrictive. That is, thetechnology according to the present disclosure can exhibit other effectsobvious to those skilled in the art from the description of the presentspecification together with or in place of the above-described effects.

Note that following configurations also belong to the technical scope ofthe present disclosure.

(1)

A medical support arm system including:

a joint state acquisition unit configured to acquire a state of a jointunit of an arm unit; and

an external force estimation unit configured to estimate an externalforce due to predetermined disturbance on the basis of a condition thatthe external force due to the predetermined disturbance is limited toone predetermined direction or a plurality of predetermined directions,and a state of the joint unit.

(2)

The medical support arm system according to (1), in which

the arm unit includes a number of torque sensors, the number beingsmaller than the number of joint units configuring the arm unit.

(3)

The medical support arm system according to (1) or (2), in which

the arm unit includes at least three or more series of joint units, andtorque sensors included in adjacent joint units of the three or morejoint units have independent degrees of freedom of each other.

(4)

The medical support arm system according to any one of (1) to (3), inwhich

the arm unit includes an encoder having six degrees or more of freedom.

(5)

The medical support arm system according to any one of (1) to (4), inwhich

the plurality of joint units configuring the arm unit includes a jointunit including an actuator and an encoder, and a joint unit including anactuator, an encoder, and a torque sensor.

(6)

The medical support arm system according to any one of (1) to (5), inwhich

the external force estimation unit estimates the external force thatacts on a predetermined observation point.

(7)

The medical support arm system according to (6), in which

the observation point includes at least one of a trocar point, a camerahead, or a distal end of an endoscope.

(8)

The medical support arm system according to (7), in which

the external force estimation unit estimates the external force actingon the observation point, and

the medical support arm system further includes a joint control unitconfigured to control the joint unit according to the external force.

(9)

The medical support arm system according to (8), in which

the observation point includes the distal end of the endoscope,

the external force estimation unit estimates the external force actingon the distal end of the endoscope, and

the joint control unit controls the joint unit according to the externalforce.

(10)

The medical support arm system according to (9), in which

the joint control unit controls the joint unit such that the arm unitmoves in a direction according to a direction of the external force orthe arm unit stops in a case where it is estimated that the externalforce has acted on the distal end of the endoscope.

(11)

The medical support arm system according to (8), in which

the observation point includes the camera head,

the external force estimation unit estimates the external force actingon the camera head, and

the joint control unit controls the joint unit according to the externalforce.

(12)

The medical support arm system according to (11), in which

the joint control unit controls the joint unit such that the arm unitmoves in a direction according to a direction of the external force in acase where it is estimated that the external force has acted on thecamera head.

(13)

The medical support arm system according to any one of (1) to (12),further including:

an output control unit configured to perform control such that an alertis output by an output device in a case where the external force exceedsa threshold value.

(14)

The medical support arm system according to any one of (1) to (12),further including:

an output control unit configured to perform control such that magnitudeof the external force or a direction of the external force is output byan output device.

(15)

The medical support arm system according to (13) or (14), in which

the output device includes at least one of a display device, a soundoutput device, or a light output device.

(16)

The medical support arm system according to any one of (13) to (15),further including:

the output device.

(17)

The medical support arm system according to any one of (1) to (16), inwhich

the disturbance includes disturbance due to tension of a light sourceand a camera cable.

(18)

The medical support arm system according to any one of (1) to (17), inwhich

a monitor included in a navigation system is connected to the arm unit,and

the external force estimation unit estimates a force received from themonitor as the external force.

(19)

The medical support arm system according to (18), further including:

the monitor.

(20)

The medical support arm system according to any one of (1) to (17), inwhich

the arm unit supports a retractor or a forceps, and

the external force estimation unit estimates a force by which theretractor or the forceps is operated by an operator, as the externalforce.

(21)

A control device including:

a joint state acquisition unit configured to acquire a state of a jointunit of an arm unit; and

an external force estimation unit configured to estimate an externalforce due to predetermined disturbance on the basis of a condition thatthe external force due to the predetermined disturbance is limited toone predetermined direction or a plurality of predetermined directions,and a state of the joint unit.

REFERENCE SIGNS LIST

-   1 Arm control system-   10 Arm device-   20 Control device-   30 Display device-   110 Arm control unit-   111 Drive control unit-   120 Arm unit-   130 Joint unit-   131 Joint drive unit-   132 Indirect state detection unit-   133 Rotation angle detection unit-   134 Torque detection unit-   140 Imaging unit-   210 Input unit-   220 Storage unit-   230 Control unit-   240 Whole body coordination control unit-   241 Arm state acquisition unit-   242 Arithmetic condition setting unit-   243 Virtual force calculation unit-   244 Real force calculation unit-   250 Ideal joint control unit-   251 Disturbance estimation unit-   252 Command value calculation unit

1. A medical support arm system comprising: a joint state acquisitionunit configured to acquire a state of a joint unit of an arm unit; andan external force estimation unit configured to estimate an externalforce due to predetermined disturbance on a basis of a condition thatthe external force due to the predetermined disturbance is limited toone predetermined direction or a plurality of predetermined directions,and a state of the joint unit.
 2. The medical support arm systemaccording to claim 1, wherein the arm unit includes a number of torquesensors, the number being smaller than the number of joint unitsconfiguring the arm unit.
 3. The medical support arm system according toclaim 2, wherein the arm unit includes at least three or more series ofjoint units, and torque sensors included in adjacent joint units of thethree or more joint units have independent degrees of freedom of eachother.
 4. The medical support arm system according to claim 1, whereinthe arm unit includes an encoder having six degrees or more of freedom.5. The medical support arm system according to claim 1, wherein theplurality of joint units configuring the arm unit includes a joint unitincluding an actuator and an encoder, and a joint unit including anactuator, an encoder, and a torque sensor.
 6. The medical support armsystem according to claim 1, wherein the external force estimation unitestimates the external force that acts on a predetermined observationpoint.
 7. The medical support arm system according to claim 6, whereinthe observation point includes at least one of a trocar point, a camerahead, or a distal end of an endoscope.
 8. The medical support arm systemaccording to claim 7, wherein the external force estimation unitestimates the external force acting on the observation point, and themedical support arm system further comprises a joint control unitconfigured to control the joint unit according to the external force. 9.The medical support arm system according to claim 8, wherein theobservation point includes the distal end of the endoscope, the externalforce estimation unit estimates the external force acting on the distalend of the endoscope, and the joint control unit controls the joint unitaccording to the external force.
 10. The medical support arm systemaccording to claim 9, wherein the joint control unit controls the jointunit such that the arm unit moves in a direction according to adirection of the external force or the arm unit stops in a case where itis estimated that the external force has acted on the distal end of theendoscope.
 11. The medical support arm system according to claim 8,wherein the observation point includes the camera head, the externalforce estimation unit estimates the external force acting on the camerahead, and the joint control unit controls the joint unit according tothe external force.
 12. The medical support arm system according toclaim 11, wherein the joint control unit controls the joint unit suchthat the arm unit moves in a direction according to a direction of theexternal force in a case where it is estimated that the external forcehas acted on the camera head.
 13. The medical support arm systemaccording to claim 1, further comprising: an output control unitconfigured to perform control such that an alert is output by an outputdevice in a case where the external force exceeds a threshold value. 14.The medical support arm system according to claim 1, further comprising:an output control unit configured to perform control such that magnitudeof the external force or a direction of the external force is output byan output device.
 15. The medical support arm system according to claim13, wherein the output device includes at least one of a display device,a sound output device, or a light output device.
 16. The medical supportarm system according to claim 13, further comprising: the output device.17. The medical support arm system according to claim 1, wherein thedisturbance includes disturbance due to tension of a light source and acamera cable.
 18. The medical support arm system according to claim 1,wherein a monitor included in a navigation system is connected to thearm unit, and the external force estimation unit estimates a forcereceived from the monitor as the external force.
 19. The medical supportarm system according to claim 18, further comprising: the monitor. 20.The medical support arm system according to claim 1, wherein the armunit supports a retractor or a forceps, and the external forceestimation unit estimates a force by which the retractor or the forcepsis operated by an operator, as the external force.
 21. A control devicecomprising: a joint state acquisition unit configured to acquire a stateof a joint unit of an arm unit; and an external force estimation unitconfigured to estimate an external force due to predetermineddisturbance on a basis of a condition that the external force due to thepredetermined disturbance is limited to one predetermined direction or aplurality of predetermined directions, and a state of the joint unit.